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Preface 


['he  purpose  of  this  thesis  was  to  determine  if  decision  theory  would  provide  a 
suitable  framework  for  an  expert  system  reasoning  methodology.  Particular  attention 
was  directed  towards  how  uncertainty  and  utility  assessments  could  be  factored  into 
a  decision  in  a  consistent  manner.  Though  the  model  presented  does  ho.ve  some 
limitations,  the  approach  nonetheless  is  valid  in  a  number  of  areas  where  rational 
thinking  and  uncertainty  are  factored  into  decision  making.  Hopefully  this  research 
has  broken  ground  for  the  development  of  a  full-scale  expert  system  developed  on 
the  concepts  of  decision  theory. 
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Abstract 

The  applicability  of  using  a  decision  theory  approach  towards  reasoning  in 
expert  systems  is  presented.  Particular  emphasis  is  placed  on  the  management  of 
uncertainty  and  how  it  can  be  consistently  factored  into  decision  making.  Using  a 
PC- based  Scheme  program  as  a  decision-analytic  reasoner,  and  a  Quattro™  gener¬ 
ated  file  composed  of  user  provided  likelihoods  as  the  knowledge  base,  this  capability 
is  demonstrated  using  an  anomalous  condition  on  board  the  GPS  satellite  as  the  test 
scenario.  The  model,  using  Jeffrey’s  Rule  as  a  manner  of  manipulating  uncertainty, 
is  able  to  effectively  capture  GPS  system  knowledge  probabilistically  in  a  manner 
that  matches  expectations  of  the  experts  as  well  as  coincides  with  the  GPS  Orbital 
Operations  Handbook.  How  rules  are  represented  in  this  type  of  system  is  discussed 
as  well  as  the  assumptions  which  go  into  the  model’s  application.  Limitations  of  the 
system  and  recommendations  for  expanding  its  uses  are  also  given. 
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DECISION-ANALYTIC  APPROACH  TO 
RULE-BASED  EXPERT  SYSTEM  DEVELOPMENT 
USING  GPS  AS  THE  MODEL 


I.  Introduction 

l.l  Background. 

In  the  realm  of  satellite  operations,  sound  and  accurate  decision  making  can 
mean  the  difference  between  saving  a  troubled  vehicle  or  losing  it.  The  recent  loss 
of  the  Soviet  Phobos  satellite  due  to  operator  error  attests  to  this  fact  (9:31).  As 
it  presently  stands,  satellite  anomaly  resolution  is  a  dynamic  and  often  probabilistic 
process  which  requires  posterior  analysis  of  the  available  telemetry  evidence  (£,)  to 
determine  hypotheses  (Hx)  of  the  anomaly’s  cause  (8:11).  Based  on  the  expected 
utility  of  a  particular  action,  a  decision  is  made.  Presently  this  process  is  performed 
in  a  heuristic  manner  based  upon  an  expert’s  examination  of  the  given  data  as  well 
as  his  beliefs  as  to  what  caused  the  problem.  This  ability  to  competently  analyze 
and  rectify  satellite  anomalies  is  gained  from  years  of  hands  on  experience.  As  the 
number  of  satellites  continues  to  grow,  the  luxury  of  this  lengthy  learning  process 
will  disappear. 

Because  the  Global  Positioning  System  (GPS)  satellite  program  is  the  first 
such  operated  by  the  Air  Force,  questions  have  been  raised  as  to  how  the  knowledge 
base  of  expertise  is  going  to  grow  as  satellite  operators  are  transferred  into  and  out 
of  the  unit.  Rule-based  expert  systems  are  looked  to  as  a  way  to  capture  the  growing 
body  of  knowledge  and  expertise.  Research  by  Knue  (1986)  and  Rampino  (1987) 
shows  this  indeed  is  a  practical  and  likely  option  for  the  Air  Force  to  pursue. 
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Expert  systems  are  knowledge-based  computer  programs  which  behave  like  ex¬ 
perts  in  chosen  domains  of  application  (2:314).  They  provide  a  practical  means  of 
building  automated  experts  in  areas  where  job  excellence  requires  consistent  reason¬ 
ing  and  rewards  practical  experience  (29:963).  They  have  proven  themselves  to  be 
quite  beneficial  in  areas  such  as  medical  diagnosis,  and  geological  exploration.  In 
particular,  MYCIN.  which  is  used  as  a  consultant  to  medical  personnel,  has  success¬ 
fully  recommended  actions  which  parallel  those  of  other  medical  experts  (4:589-596). 
Fable  l.l  outlines  the  different  areas  rule-based  technology  has  been  successfully  ap¬ 
plied. 


Table  1.1.  Rule-Based  System  Applications 


Problem 

System  Functions  j 

Equipment  maintenance 

Diagnose  faults  and  recommend  repairs 

Component  selection 

Elicit  requirements  and  match  parts 

- 

catalog 

Computer  operation 

Analyze  requirements;  select  and 
operate  software 

Product  configuration 

Elicit  preferences  and  identify  parts 
that  satisfy  constraints 

Troubleshooting 

Analyze  situation,  suggest  treatments, 
and  prescribe  preventative  measures 

Process  control 

Spot  problematic  data  and  remedy 
irregularities 

Quality  assurance 

Assess  task,  propose  practices,  and 
enforce  requirements 

Most  expert  systems  are  built  upon  a  series  of  conditional  rules  which  govern 
how  knowledge  is  represented  and  managed.  These  rules  are  written  in  a  "‘If  A  then 
B"  format.  This  means  that  if  condition  A  is  true,  then  conclude  B.  However,  incon¬ 
sistencies  arise  when  the  various  systems  try  to  represent  incomplete  or  uncertain 
knowledge.  In  MYCIN,  an  example  of  this  uncertain  knowledge  would  be  written 
into  a  rule  as  follows: 


IF: 

1)  The  gram  stain  of  the  organism  is  gramneg,  and 

2)  The  morphology  of  the  organism  is  rod,  and 

3)  The  aerobicity  of  the  organism  is  anaerobic 

Then:  There  is  suggestive  evidence  (.6)  that  the  identity  of  the  organism  is 
bactroides  (4:71 ). 

How  this  uncertainty  is  managed  and  how  rules  are  combined  in  order  to  rec¬ 
ommend  particular  courses  of  action  varies  from  system  to  system.  This  thesis  will 
examine  a  decision-analytic  approach  towards  managing  uncertainty  in  expert  svs- 
tems. 

1.2  Problem  Statement 

The  purpose  of  this  thesis  is  to  demonstrate  the  applicability  of  using  a  Bayesian 
decision-analytic  approach  to  handle  uncertainty  and  utility  in  expert  system  devel¬ 
opment.  The  research  will  focus  specifically  on  how  this  approach  can  be  used  to 
diagnose  anomalous  conditions  aboard  GPS  satellites. 

1.3  Scope 

The  scope  of  this  thesis  will  determine  if  the  GPS  contingency  procedures 
and  system  knowledge  can  adequately  be  represented  in  a  decision-analytic  format. 
Particular  emphasis  will  concern  the  incorporation  and  management  of  second-order 
uncertainty  and  its  effects  on  anomaly  resolution.  The  thesis  is  structured  as  follows. 
Chapter  II  is  a  literature  review  which  examines  uncertainty  representation  in  expert 
sytems.  Reasons  why  decision  theory  is  a  viable  option  will  be  discussed.  Chapter  III 
will  outline  the  methodology  and  the  steps  needed  to  fulfill  the  research  objective. 
Chapter  IV  gives  background  data  as  to  why  the  GPS  program  was  selected  and 
what  successful  application  of  the  model  in  the  satellite  operations  domain  could 
mean.  Chapter  V  discusses  the  assumptions  the  decision-analytic  model  is  based  on 
and  how  it  differs  from  other  rule  bases  in  its  representation  of  knowledge.  Chapter 
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VI  explains  how  to  apply  the  method  to  problems  in  general  while  Chapter  VII 
will  apply  the  model  to  a  particular  GPS  problem  .  Chapter  VIII  will  summarize 
conclusions  based  on  results  of  Chapter  VII,  and  give  recommendations  as  to  the 
overall  viability  of  this  approach  towards  knowledge  representation  in  rule-based 
expert  systems  in  general. 
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II.  Literature  Review 


3.1  Introduction 

A  major  point  of  contention  among  leaders  in  the  Artificial  Intelligence  (AI) 
community  is  how  uncertainty  should  be  expressed  and  managed  in  expert  systems. 
There  are  four  major  schools  of  thought,  each  with  differing  views  concerning  uncer¬ 
tain  reasoning.  These  areas  are  classic  probability  or  Bayesian  reasoning,  Zadeh’s 
fuzzy  set  theory,  Shortliffe’s  certainty  factors,  and  Dempster-Shafer’s  upper  and  lower 
probabilities  (16:2).  The  goal  of  the  literature  review  is  to  examine  the  competing 
strategies  in  handling  uncertainty  in  expert  systems.  A  brief  overview  of  how  rea¬ 
soning  is  accomplished  is  given  and  then  the  general  theory  of  each  methodology 
is  outlined  as  well  as  its  relative  strengths  and  criticisms.  Because  this  thesis  will 
focus  on  a  decision-analytic  approach  to  expert  system  reasoning,  this  method  is  also 
reviewed  and  its  suitability  to  the  stated  objective  is  given. 

3.3  Reasoning  Under  Uncertainty 

The  presence  of  uncertainty  in  reasoning  systems  is  caused  by  a  variety  of 
sources:  the  reliability  of  the  information,  the  inherent  imprecision  of  the  repre¬ 
sentation  language  in  which  the  information  is  conveyed,  the  incompleteness  of  the 
information,  and  the  aggregation  or  summarization  of  information  from  multiple 
sources  (29:854).  How  to  represent  this  quality  of  belief  is  an  ever-present  problem 
in  the  reasoning  process  of  expert  systems;  a  problem  that  has  yet  to  be  solved  in  a 
universally  accepted  manner. 

In  logical  systems,  reasoning  is  performed  in  two  manners — modus  ponens  and 
modus  tollens.  Considering  two  propositions  p  and  q  and  assuming  the  implication 
p  =>  q  to  be  true,  modus  ponens  allows  the  deduction  of  truth  of  q  from  the  truth 
of  p  (see  Figure  2.1).  Conversely,  modus  tollens  allows  the  deduction  of  falsity  of 
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p  from  the  falsity  of  q.  These  statements  assume  that  the  propositions  are  either 
true  or  false.  This  exactness  is  lost  when  dealing  with  uncertainty  concerning  the 
propositions.  Consequently,  given  a  proposition  p  and/or  an  implication  p  =>  q 
whose  truth  cannot  be  established  definitively,  (i.e.,  some  other  value  besides  0  or  1) 
what  can  be  said  about  q?  (11:5) 


Modus  Ponens 

p  ==>  q 

p 

q 

Modus  Tollens 

p  =>  q 

-’p 

Figure  2.1.  Logic  Reasoning 

In  situations  like  these,  where  knowledge  is  not  definitive,  uncertainty  must 
be  introduced  in  a  manner  that  is  consistent,  consistent  in  the  sense  that  the  logic 
used  propogates  throughout  the  system  in  a  coherent  manner.  The  different  schemes 
previously  mentioned  are  methods  which  propose  how  expert  systems  should  deal 
with  this  uncertainty. 
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2.3  Probability  Theory 

In  his  paper,  "The  Probability  Approach  to  the  Treatment  of  Uncertainty  in 
Artificial  Intelligence  and  Expert  Systems,”  Lindley  points  to  the  inevitability  and 
practicality  of  using  probability  in  reference  to  uncertain  events.  With  its  sound 
theoretic  basis.  Lindley  convincingly  argues  that  probability  theory  is  the  only  way 
to  adequately  discuss  uncertainty  (19:17-20).  Henrion  reiterates  this  position  and 
notes  that  probability  theory  can  be  derived  from  a  set  of  simple  axioms  governing 
rational  decision-making  under  uncertainty.  It  is  these  axioms  which  form  the  basis 
of  decision  theory  (12:2). 

As  it  applies  to  reasoning  in  expert  systems,  probability  offers  definitive  ad¬ 
vantages  over  non-probabilistic  measures  of  uncertainty.  First,  it  provides  a  sound 
theory  for  using  computed  uncertainties  in  decision  making.  Second,  it  provides 
an  operational  definition  for  the  likelihood  (probability)  of  an  event  in  terms  of  a 
person’s  behavior.  Third,  probabilistic  schemes  provide  well-known  ways  of  incor¬ 
porating  empirical  data.  And  fourth,  “there  are  well-developed  methods  for  evalu¬ 
ating  judged  or  computed  probabilities  by  comparison  with  empirical  frequencies,  in 
terms  of  accuracy,  resolution  and  calibration”  (12:2).  No  non-probabilistic  measures 
of  uncertainty  offer  these  advantages.  Lindley,  who  generalized  a  scoring  system  to 
determine  how  good  of  an  assessment  different  uncertainty  methods  are.  points  out 
that  none  do  better  than  probability  (19:27)  (20:18). 

2.3.1  Concepts  of  Probability  The  application  of  probability  theory  requires 
the  prescription  or  the  determination  of  some  probability  distribution  over  a  set  of 
objects  (11:3).  Each  object  is  assigned  a  value  between  0  and  1  such  that  for  each 
object  i  the  sum  of  the  probability  of  i  (P,)  would  equal  one  (^2  P,  =  1)-  Bayes’ 
theorem,  which  was  developed  by  the  Reverend  Thomas  Bayes,  is  the  cornerstone 
upon  which  the  foundation  of  probability  reasoning  rests.  The  theorem  provides  a 
method  in  which  different  probabilities  can  be  combined  in  a  consistent  and  coherent 
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manner. 


According  to  a  Bayesian,  probability  measures  a  person’s  degree  of  belief  in  a 
particular  event  or  proposition,  given  the  information  currently  known  to  that  per¬ 
son.  Mathematically,  the  theorem  states  that  if  A  and  B  are  two  events,  and  that 
the  P(A)  is  not  zero,  P(B|A)  =  P(  A|B)*P(B)/P(A).  (This  is  read  as  the  probability 
of  event  B  given  event  A  is  known.)  Knue  points  out  that  there  are  only  a  few 
expert  systems  which  strictly  utilize  sound  Bayesian  probability  theory  in  their  rea¬ 
soning  (10:3).  This  is  surprising  considering  that  Henrion  claims  that  anyone  who 
violates  the  law  of  probability  and  acts  on  incoherent  probabilities,  is  liable  to  suffer 
demonstrable  loss  (7:2).  Lindley’s  explanation  of  the  “Dutch  Book-’  demonstrates 
the  necessity  of  having  such  a  coherent  theory  (12:51). 

2.3.2  Criticisms  of  Probability  There  are  several  criticisms  against  using 
Bayesian  probability  theory  in  expert  systems.  Below  are  the  major  ones  as  pointed 
out  by  Henrion  (12),  Goicoechea  et  al.  (10),  and  Groothuizen  (11): 

(1)  The  rule  requires  all  the  hypotheses  to  be  disjoint  and,  in  a  large  expert 
system,  dividing  the  solution  space  into  mutually  exclusive  subsets  may  be  expensive; 
(10:560) 

(2)  A  large  amount  of  statistical  data  is  needed  in  the  form  of  all  P(ej|/i,); 

(11:3) 

(3)  In  the  event  of  altering  the  probability  of  an  event  in  the  system  (by  adding 
or  removing  hypotheses)  ail  the  probabilities  would  need  to  be  recalculated;  (10:560) 

(4)  There  is  no  parameter  that  insures  the  set  of  probabilities  built  into  an 
expert  system  is  consistent  and  coherent;  for  example,  the  product  P(A|B)P(B)  may 
or  may  not  be  equal  to  P(B|A)P(A);(10:560) 

(5)  In  realistic  situations  evidentiary  information  can  quickly  translate  into 
very  long  sums  and  products  of  conditional  and  marginal  distributions  requiring 
substantial  storage  and  computing  resources;)  10:560) 
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(6)  Ignorance,  vagueness,  or  second-order  uncertainty  cannot  be  represented: 

[ 12:4) 

(7)  In  this  approach:  P(not  h,  |  e,)  =  1  -  P(/i,  |  e}),  that  is  evidence  supports 
both  h,  and  (not  h,),  although  not  to  the  same  degree.  This  is  intuitively  very 
peculiar  (i.e.,  this  not  how  humans  think);  (11:3) 

(8)  It  is  difficult  to  combine  evidence  if  it  is  not  independent.  Even  then,  it  is 
not  necessarily  true  that  more  evidence  always  gives  more  belief  in  a  conclusion  as 
probability  theory  tells  us.  (11:3) 

2.4  Dempster-Shafer  Theory  of  Evidence  (Belief  Theory) 

The  Dempster-Shafer  theory  of  evidence  (Dempster,  1967;  Shafer  1976)  is  basi¬ 
cally  a  set-theoretic  generalization  of  Bayesian  Theory  (10:560).  It  involves  establish¬ 
ing  upper  and  lower  probabilities  which  bound  the  subjective  Bayesian  probability 
for  a  particular  hypothesis  (16:23).  Knue  reports  that  this  bounding  gives  some 
flexibility  to  inferences  made  in  expert  systems  which  use  this  technique  (16:23).  As 
noted  by  Gordon  and  Shortliffe  (1984)  the  advantage  of  Dempster-Shafer  theory  over 
other  approaches  in  handling  uncertainty  is  its  ability  to  model  the  narrowing  of  the 
hypothesis  set  with  the  accumulation  of  evidence  (4:272).  Shortliffe  believes  this  pro¬ 
cess  characterizes  diagnostic  reasoning  in  medicine  and  expert  reasoning  in  general. 
Shafer's  Belief  theory  has  received  considerable  attention  in  recent  years  because  it 
has  a  sound  theoretical  basis,  it  subsumes  Bayesian  theory,  it  is  able  to  represent 
ignorance,  and  it  allows  beliefs  from  several  sources  to  be  treated  symmetrically  and 
pooled  together  (18:238). 

2,4-1  Concepts  of  Dempster-Shafer.  Dempster-Shafer  theory  is  based  upon 
a  concept  known  as  belief  functions.  A  hypothesis  is  given  a  number  in  the  range 
of  [0,1]  to  indicate  belief  in  the  hypothesis  given  some  evidence  (4:275).  “For  every 
piece  of  evidence,  a  basic  probability  function  (bpf)  m:  P(0)  ->  [0, 1  j  is  defined  such 
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that  m  [0]  =  0  and  £  m[A]  =  l'1  (11:9).  0  represents  the  set  of  all  hypotheses  and 
P(0)  is  the  power  set  of  0  which  is  summed  over  all  subsets,  A,  of  0.  m[A)  measures 
that  portion  of  the  total  belief  committed  precisely  to  A.  Unlike  probability  theory, 
mf.-U]  ^  1  -  m[A].  (The  belief  and  the  disbelief  do  not  necessarily  have  to  sum  to 
one).  The  total  belief  in  A  is  measured  by  the  belief  function  Bel:  Bel  [A]  =  Y1  m[B] 
(11:9-10).  After  combining  different  evidences,  a  belief  interval  is  defined  which 
represents  the  strength  of  belief  and  disbelief  in  a  hypothesis.  See  Groothuizen  (11) 
or  Knue  (16)  for  further  explanation  on  how  evidence  is  combined  and  managed  in 
expert  systems. 

2.4.2  Criticisms  of  Dempster-Shafer.  There  are  several  criticisms  against 
Dempster-Shafer  belief  functions.  Goicoechea  et  al.  (10)  and  Groothuizen  (11) 
point  out  some  problems  with  the  theory  that  need  to  be  addressed  in  greater  detail: 

(1)  Computation  of  combined  bpf’s  is  very  time  consuming;  consequently  in 
realistic  cases  a  long  chain  of  inferences  may  make  the  theory  very  inconvenient  and 
expensive  to  use  because  of  the  increasing  complexity  in  the  structure  of  the  core  of 
the  belief  functions;  (11:10) 

(2)  The  definition  of  these  combined  bpf’s  indicates  some  kind  of  assumed 
independence;  (11:10) 

(3)  Dempster’s  rule  of  combination  cannot  be  applied  in  situations  where  there 
are  considerable  disagreements  among  the  evidence,  that  is,  when  cores  of  two  belief 
functions  are  disjoint  (i.e.,  contradictory);  (10:560) 

(4)  The  numerical  stability  of  the  theory  needs  to  be  analyzed  further;  in 
some  cases,  small  variations  in  the  basic  probability  assignments  can  produce  a  large 
variation  in  the  results.  (10:560) 
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2.5  Fuzzy  Set  Theory  (Necessity  and  Possibility  Theory) 


Fuzzy  set  theory,  which  was  developed  by  Zadeh  (1965),  is  another  attempt  to 
quantify  uncertainty  and  imprecision  in  expert  system  reasoning.  Fuzzy  reasoning, 
which  introduces  the  notions  of  necessity  and  possibility,  looks  at  the  world  as  not 
only  being  black  and  white,  but  as  a  continuum  of  grey  (11:12). 

Fuzzy  sets  differ  from  the  classically  defined  set  where  a  member  can  either  be 
in  the  set  or  out  of  it,  but  not  both.  A  fuzzy  set  is  a  class  of  objects  which  hcve 
various  grades  of  membership  in  that  set  (16:17).  The  degree  of  membership  is  always 
a  real  number  between  0  and  1.  and  it  measures  the  extent  to  which  an  element  is 
in  a  fuzzy  set.  In  ordinary  set-theoretic  terms,  the  degree  of  membership  measures 
the  plausibility  of  an  element  being  in  a  particular  set  (28:5).  Consequently  fuzzy 
sets  are  not  mutually  exclusive  or  exhaustive  (16:17).  If  an  element’s  membership  is 
restricted  to  either  1  or  0,  fuzzy  sets  reduce  to  normal  sets.  As  a  result,  normal  set 
theory  is  a  special  case  of  fuzzy  set  theory.  A  detailed  analysis  for  defining  fuzzy  set 
membership  functions  can  be  found  in  Zimmerman  (33)  and  Schmuker  (28). 

2.5.1  Concepts  of  Fuzzy  Set  Theory.  Fuzzy  set  theory  was  developed  in  order 
to  capture  the  vagueness  inherent  in  some  linguistic  terms.  This  vagueness  is  evident 
when  you  try  to  quantify  concepts  like  hot  or  cold,  where  hot  in  the  context  of  weather 
is  different  than  hot  when  used  to  describe  a  fusion  reactor.  Groothuizen  (11)  gives 
a  good  example  using  the  word  tall.  The  statement  “John  is  tall”  can  be  represented 
by:  length  (John)  £  A  where  length  is  known  exactly  and  A  is  a  fuzzy  set,  e.g.. 


uA(i) 


0  if  x  <  170cm 

1  if  x  >  190cm 

or  otherwise 


Goicoechea  et  al.  point  out  that  there  have  been  a  number  of  applications  of 
fuzzy  logic  to  expert  systems,  including  SP  II  and  Reveal  (10:560). 


3.5.2  Criticisms  of  Fuzzy  Set  Theory.  Below  are  some  criticisms  against  us¬ 
ing  fuzzy  set  theory  in  expert  systems: 

(1)  The  maximum  and  minimum  rules  for  disjunction  and  conjunction  may 
cancel  valuable  information  when  fuzzy  individual  assignments  to  various  pieces  of 
evidence  include  one  assignment  that  is  very  close  to  zero;  (10:560) 

(2)  Membership  functions  are  context-sensitive;  for  example,  a  "small"  building 
can  be  bigger  than  a  "big"  house;  generic  membership  functions,  if  applied  blindly, 
can  lead  to  misleading  results;  (10:560) 

(5)  Computational  and  storage  requirements  can  be  large  whenever  individ¬ 
ual  membership  functions  are  non-linear  or  non-trivial;  discrete  representation  of 
functions  is  expensive;  (10:560) 

(4)  Behavioral  studies  are  lacking  that  would  shed  some  light  into  the  task  of 
knowledge  representation  via  membership  functions  by  the  user.  (10:560) 

3.6  \1YCI\  Certainty  Factors  (Confirmation  Theory) 

Certainty  factors,  which  were  developed  to  manage  uncertainty  in  MYC'IN. 
the  medical  consultant  rule-based  expert  system,  are  the  most  popular  of  all  the 
nun-probabilistic  techniques.  Initially  based  on  Bayes’  Theorem,  certainty  factors 
were  developed  after  the  Bayesian  method  for  updating  evidence  was  considered 
inadequate.  Buchanan  and  Shortliffe  (4)  give  a  thorough  explanation  of  how  and 
why  certainty  factors  were  decided  upon  and  developed. 

The  basic  principles  behind  certainty  factors  are  the  measurements  of  "belief" 
and  "disbelief  of  a  particular  hypothesis.  Based  upon  Shortliffe's  rules  of  combina¬ 
tion,  a  number  is  reached  which  embodies  the  total  belief  in  a  particular  hypothesis 
given  all  known  evidence.  Knue  ( 16)  does  a  good  job  of  explaining  the  rules  of  com¬ 
bination  for  certainty  factors  as  well  as  discussing  some  of  its  applications.  Below  is 
a  guick  overview  of  the  system  as  presented  by  Goicoecea  et  a!. 


2.6.1  Concepts  of  Certainty  Factors.  MYCLVs  theoretical  framework  in¬ 
cludes  terminology  such  as  ’’measure  of  belief",  denoted  MB,  ‘‘measure  of  disbelief’, 
denoted  MD,  and  '‘certainty  factors”,  CF.  Formally  these  are  defined  as:  MB(H.E) 
is  the  measure  of  the  belief  in  the  hypothesis  H,  given  evidence  E;  MD(H.E)  is 
the  measure  of  disbelief  in  the  hypothesis  H,  given  evidence  E,  and  CF(H.E)  = 

\1B<H.E)  -  MD(H.E). 

Since  MB(H,E)  is  a  number  between  0  and  1,  and  MD(H,E)  is  also  a  number 
between  0  and  L,  the  cert -duty  factor  CF(H,E)  is  a  number  between  -1  and  +1.  A 
positive  CF  indicates  there  is  more  reason  to  believe  a  hypothesis  than  disbelief  it.  A 
negative  CF  indicates  that  a  hypothesis  should  be  rejected  more  than  confirmed.  A 
CF  of  0  is  a  "don't  know”  value  which  indicates  that  the  hypothesis  is  independent  of 
the  evidence  or  equally  supported  (MB)  and  disconfirmed  ( MD)(  10:561 ).  Measures 
of  belief  are  incorporated  into  the  rules  of  the  knowledge  base  and  are  used  to 
propogate  evidence  towards  a  hypothesis.  Though  originally  developed  with  medical 
diagnosis  in  mind,  there  are  several  non-medical  expert  systems  which  use  certainty 
factors  as  the  means  of  representing  uncertainty. 

2.6.2  Criticisms  of  Certainty  Factors.  The  following  are  some  criticisms  against 
certainty  factors: 

( 1 )  The  definitions  for  combining  the  rules  are  rather  arbitrary.  Consequently, 
in  some  instances,  results  match  an  expert’s  intuitive  feeling  and  in  other  cases, 
unexpected  and  unwanted  results  are  yielded;  (119) 

(2)  Combining  rules  assumes  independence  of  the  evidences;  (11:9) 

(3)  It  is  does  not  have  a  strong  theoretic  basis;  (10:562) 

(4)  The  semantics  of  the  CF,  i.e. ,  the  interpretation  of  the  number  (ratio  of 
probability,  combination  of  utility  values  and  probability);(29:857) 

(5)  rnability  to  distinguish  between  ignorance  and  conflict,  both  which  are 
represented  by  a  zero  CF.  (29:857) 
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This  concludes  the  overview  of  some  of  the  competing  methodologies  which 
profess  to  being  able  to  represent  uncertainty  reasoning  in  expert  systems.  Because 
id'  the  brevity  devoted  to  each  method,  the  referenced  articles  should  be  consulted 
for  a  more  thorough  understanding  of  particular  concepts. 

2.7  Decision-Theoretic  Approach  (Decision  Theory) 

As  discussed  by  Hollenga  and  Morlan  in  their  paper,  “A  Decision-Theoretic 
Model  for  Constructing  Expert  Systems,”  expert  systems  should  use  decision  anal- 
vsis  as  the  methodology  for  dealing  with  uncertainty  because  of  its  sound  theoretic 
base  (13:2).  Composed  from  the  axioms  of  probability  and  utility,  decision  theory 
provides  a  framework  for  coherent  assignment  of  beliefs  with  incomplete  information 
(  probability),  while  introducing  a  set  of  principles  for  consistency  among  preferences 
and  decisions  (utility). 

2.7.1  Overview.  The  following  paragraphs  will  summarize  why  a  decision- 
analytic  approach  offers  more  than  the  previously  listed  alternatives  for  representing 
knowledge  in  expert  systems.  The  strengths  of  probability  and  utility  theory  are 
briefly  reviewed,  and  how  the  synergistic  combination  of  the  two  in  decision  theory 
is  both  a  viable  method  of  reasoning  and  an  accurate  measure  of  uncertainty.  Lastly 
the  suitability  of  the  approach  for  representing  knowledge  about  the  GPS  satellite 
program  is  discussed. 

2. 7.1. 1  Why  Decision  Theory  uses  Probability  for  Representatmg  Un¬ 
certainty.  There  are  a  number  of  reasons  why  decision  theory  uses  probability  as 
its  representation  of  uncertainty.  Below  is  a  list  as  presented  by  Horvitz  et  al.  of 
fundamental  properties  that  are  considered  intuitively  desireable  for  measuring  belief 
in  the  truth  of  a  proposition. 
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1.  Clarity:  Propositions  should  be  well  defined. 

2.  Scalar  continuity:  A  single  real  number  is  both  necessary  and  sufficient  for 
representing  a  degree  of  belief  in  a  proposition. 

3.  Completeness:  A  degree  of  belief  can  be  assigned  to  any  well-defined  proposi¬ 
tion. 

4.  Context  dependency:  The  belief  assigned  to  a  proposition  can  depend  on  the 
belief  in  other  propositions. 

5.  Hypothetical  conditioning:  There  exists  some  function  that  allows  the  belief 
in  a  conjunction  or  propositions,  B(X  A  F),  to  be  calculated  from  the  belief 
in  one  proposition  and  the  belief  in  the  other  proposition  given  that  the  first 
proposition  is  true.  That  is,  B(X  A  Y)  —  f[B(X\Y),  B(Y)\. 

6.  Complementarity:  The  belief  in  the  negation  of  a  proposition  is  a  monotoni- 
cally  decreasing  function  of  the  belief  in  the  proposition  itself. 

7.  Consistency:  There  will  be  equal  belief  in  propositions  that  are  logically  equiv¬ 
alent. 

Researchers  have  demonstrated  that,  taken  together,  these  properties  logically 
imply  that  the  measure  of  belief  must  satisfy  the  axioms  of  probability  theory. “...  If 
one  accepts  these  intuitive  properties  as  desirable,  one  must  then  accept  probabilities 
as  a  desirable  measure  of  belief.” ( 14:5)  It  is  for  these  reasons  Lindley  (19)  argues 
that  probability  is  the  only  satisfactory  description  of  uncertainty. 

Using  these  properties  as  a  gold  standard,  alternative  methods  for  representing 
uncertainty  can  be  judged  in  terms  of  which  of  the  principles  they  reject.  For  exam¬ 
ple,  Dempster-Shafer  theory  rejects  the  property  of  completeness,  denying  that  it  is 
possible  to  assign  a  belief  to  any  well-defined  proposition.  Fuzzy-set  theory  rejects 
clarity ,  allowing  linguistic  imprecision  in  the  definition  of  propositions.  Certainty 
factors  violate  consistency,  while  other  methods  reject  scalar  continuity ,  arguing 
that  a  single  number  is  insufficently  rich  to  represent  belief  (14:5). 


Concerning  the  critics  of  probabilistic  methods  in  expert  systems,  who  nonethe¬ 
less  want  a  numerical  measure  that  reflects  the  state  of  a  system’s  knowledge  about, 
or  attitude  toward,  a  proposition’s  status,  two  problems  arise.  First,  if  they  are  not 
measuring  probability,  what  are  they  measuring?  It  is  not  enough  to  associate  a 
word  ("confidence,”  "certainty,”  "degree  of  belief,”  "degree  of  confirmation,”  etc.) 
and  hope  that  semantically  it  expresses  the  user’s  underlying  intent.  There  is  a 
well-developed  theory  of  statistics,  with  interpretations  for  the  precise  meaning  of 
"probability”  as  well  as  proven  calculi  which  govern  its  use.  Any  competing  theory 
will  have  to  present  the  same  thing:  not  merely  a  name,  but  a  clear  analysis  of 
what  is  being  measured,  how  it  is  different  from  a  probability,  why  people  should 
nonetheless  be  interested  in  it,  and  how  people  can  feel  certain  that  computations 
used  to  derive  the  numbers  in  fact  compute  anything  worth  value.  To  date,  most  of 
the  work  in  these  directions  remains  suggestive  rather  than  persuasive  (29:846). 

2.7. 1.2  Utility  Theory  Provides  a  Rational  Framework  for  Reasoning. 
Utility  theory  is  based  on  a  set  of  simple  axioms  or  rules  concerning  choices  under 
uncertainty.  Just  like  the  axioms  of  probability  theory,  they  are  fairly  intuitive. 
These  axioms  provide  a  model  of  how  a  rational  man  makes  his  decisions  when  faced 
with  uncertainty.  For  the  purpose  of  decision  theory,  the  most  important  thing  to 
note  is  that  the  axioms  of  coherence  imply  that  a  person  should  make  decisions  in 
such  a  manner  as  to  maximize  the  expected  utility  (EU).  If  one  accepts  the  axioms 
of  coherence,  he  must  accept  the  EU  criterion,  because  it  follows  logically  from  the 
axioms. 

The  following  is  a  brief  summarization  of  the  axioms  of  coherence,  as  presented 
by  Winkler  (32).  See  Winkler  for  further  details  or  clarification. 

Axiom  1:  Given  any  two  payoffs  Rl  and  R2,  you  can  decide  whether  you  prefer 
Rl  to  R2,  R2  to  Rl,  or  you  can  be  indifferent  between  Rl  and  R2.(orderability) 
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Axiom  2:  If  you  prefer  payoff  Rl  to  R2  and  you  prefer  R2  to  R3,  then  you 
must  prefer  Rl  to  R3  (transivity  of  preferences).  This  is  a  very  important  element 
in  the  theory  of  subjective  probability  and  utility. 

Axiom  3:  If  you  prefer  Rl  to  R2  and  R2  to  R3,  then  you  can  find  some 
probability  value  p,  such  that  a  p-mixture  of  Rl  and  R3  is  preferred  to  R2;  you  can 
find  some  other  value  of  p  such  that  R2  is  preferred  to  a  p-mixture  of  Rl  and  R3: 
and  finally  you  can  find  yet  another  value  of  p  such  that  you  are  indifferent  bet.veen 
R2  and  a  p-mixture  of  Rl  and  R3. 

Axiom  4:  If  Rl  is  preferred  to  R2  and  R?  is  some  other  payoff  value,  then  any 
p-mixture  of  Rl  and  R3  is  preferred  to  the  same  p-mixture  of  R2  and  R3  (where  p 
value  is  the  same). 

Axiom  5:  If  you  are  ind’fferent  between  Rl  and  R2,  then  they  may  be  sub¬ 
set,  ued  for  each  other  as  payoffs  in  any  decision  making  problem. 

Axiom  6:  If  Rl  is  preferred  to  R2,  then  a  p-mixture  of  Rl  and  R2  is  preferred 
to  a  q-mixture  of  Rl  and  R2  if  and  only  if  p  >  q. 

The  six  axioms  of  coherence  appear  intuitively  reasonable  and  as  such  should 
be  adhered  to  in  rational  decision  making.  It  seems  counterintuitive  and  illogical  to 
develop  an  expert  system  which  does  not  reason  coherently. 

2.1  A. 3  Sound  Recommendations  Under  Uncertainty.  Decision  theory 
does  not  claim  to  provide  a  description  of  how  people  actually  behave  when  reason¬ 
ing  under  uncertainty,  but  how  they  should.  Horvitz  et  al.  refer  to  studies  which 
have  demonstrated  that  people  frequently  do  not  behave  in  accordance  with  deci¬ 
sion  theory.  In  fact,  characteristic  (and  often  costly)  biases  exhibited  in  intuitive 
judgement  are  part  of  the  justification  for  applying  decision  theory  to  assist  peo¬ 
ple  in  making  decisions.  While  much  can  be  done  to  determine  the  probabilities, 
structure  the  values,  and  assess  the  alternatives,  bad  outcomes  still  can  result  from 
rational  choices.  Alternatively,  a  random  or  poor  selection,  may  turn  out  to  be  quite 
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tortuitous.  Such  is  the  nature  of  acting  under  incomplete  information.  Nonetheless, 
decision  theory  strives  for  good  decisions  that  lead  to  better  outcomes  on  average 
(14:9). 


2.7.2  Suitability  of  GPS.  Though  opponents  argue  the  inadequacy  of  the 
decision-analytic  method  because  ihey  feel  ;t  requires  toe  much  prior  inforrnaf /a 
criticism  levied  at  all  probability-based  systems),  it  appears  that  the  GPS  satellite 
program  provides  a  suitable  environment  in  which  this  information  can  be  ascer¬ 


tained.  As  for  disagreements  with  the  chosen  methodology,  Henrion  (12)  effectively 
argues  that  several  of  the  criticisms  levied  against  probabilistic  reasoning  in  ex¬ 
pert  systems  are  unwarranted  and  untrue.  In  particular,  he  references  the  work  of 
Cheeseman  ( 1985)  and  Spiegelhalter  (1986)  who  show  that  ignorance,  vagueness,  and 
second-order  uncertainty  can  be  represented  in  a  sound  probabilistic  manner  (12:5). 
Pearl’s  (24)  Bayesian  networks  demonstrate  that  probabilistic  representations  of  in¬ 
formation  do  not  require  the  massive  amounts  of  data  that  many  claim.  Snow’s  (30) 
work  reiterates  this  point.  As  for  criticisms  concerning  the  intractability  of  the  the¬ 
ory,  Kim  and  Pearl’s  method  for  propagating  uncertain  evidence  through  a  Bayes’ 
network  show  that  it  is  not  necessary  to  recalculate  all  probabilities  as  new  infor¬ 


mation  is  acquired — only  those  probabilities  directly  related  to  the  new  information 
(12:3). 


Concerning  complaints  associated  with  knowledge  acquisition,  it  appears  as  if 
the  GPS  satellite’s  extensive  historical  database  make  computing  prior  probabilities 
a  statistical  problem  while  utility  assessments  can  be  provided  by  expert  opinion.  As 
for  its  applicability  to  expert  system  reasoning,  Breese  and  Fehling  point  out  that  in 
situations  where  the  cost  of  an  error  is  important,  probabilistic  reasoning  founded 
upon  decision  theory  is  the  preferred  method  of  uncertainty  reasoning  (3:33).  Kala- 
gananam  and  Henrion  (15),  using  motorcycle  repair  as  the  model,  have  demonstrated 
the  advantages  of  using  a  decision-analytic  approach  versus  a  heuristic  approach  for 
sequential  diagnosis.  Edmond’s  (8)  study  concerning  satellite  anomaly  resolution 
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also  confirms  this  point,  while  Spiegelhalter  (31)  has  also  shown  the  utility  of  this 
approach  in  medical  diagnosis. 

As  for  the  difficulty  of  generating  rules  which  capture  this  methodology,  Hol- 
lenga's  (13)  proposed  method  of  developing  rules  from  a  decision  analytic  action 
diagram  appears  promising  and  warrants  further  study. 

Overall,  the  decision- th^oietic  approach,  unlike  many  alternative  formalisms 
of  uncertainty,  provides  a  coherent  prescription  for  choosing  actions  and  meaningful 
guarantees  of  the  quality  of  these  choices  (29:55). 

2.8  Summary 

This  literature  review  has  given  some  background  on  what  the  principle  method¬ 
ologies  are  for  dealing  with  uncertainty  in  expert  systems.  Though  this  list  is  by  no 
means  exhaustive,  Bayesian  probability,  Demster-Shafter  belief  functions,  Certainty 
Factors,  and  Fuzzy  Set  Theory  represent  a  few  of  the  more  prominent  theories.  Al¬ 
though  successful  in  limited  applications,  each  method  still  has  criticism  as  to  its 
overall  effectiveness  in  varied  fields. 

In  the  section  concerning  the  decision  theoretic  approach  to  handling  uncer¬ 
tainty,  several  points  were  made  that  confirmed  decision  analyses  is  a  valid  approach 
to  uncertainty  reasoning  in  expert  systems.  For  one,  the  decisions  made  by  this 
probabilistic  method  can  be  communicated  in  intuitively  meaningful  terms.  Two, 
the  assumptions  leading  to  these  decisions  can  be  traced  back  with  ease  and  clar¬ 
ity.  And  three,  the  ability  to  assess  the  utility  of  certain  decisions,  represents  value 
judgements  about  the  preference  and  desirability  of  actions  in  the  context  of  their 
perceived  worth  (29:56).  Though  not  above  reproach,  decision  theory  can  nonethe¬ 
less  rectify  the  shortcomings  of  other  uncertainty  methods  by  providing  a  set  of  well 
founded  principles  (3:36).  The  next  chapter  will  discuss  the  methodology  used  in 
this  thesis. 


III.  Methodology 


3. 1  Introduction 

This  chapter  outlines  the  methodology  used  in  the  thesis.  Decision  theory 
is  the  framework  of  the  research  with  particular  emphasis  on  subjective  Bayesian 
Reasoning.  If  unfamiliar  with  the  major  concepts  of  decision  theory  or  Bayesian 
reasoning,  the  following  references  give  adequate  presentations  of  major  concepts 
these  theories  entail.  See  Horvitz  et  al.(  14),  Sharpiro  (29),  and  Winkler  (32). 

The  following  paragraphs  address  the  approach  needed  for  satisfying  the  sub¬ 
objectives  of  the  thesis.  It  is  the  framework  of  these  sub-objectives  that  form  the 
scope  of  this  thesis.  Consequently,  in  order  to  determine  the  feasibility  of  using  a 
decision-analytic  approach  in  developing  rule-based  expert  systems,  each  area  will 
be  examined.  The  addressed  areas  are  as  follows: 

1.  The  suitability  of  the  decision-analytic  approach; 

2.  The  present  methodology  behind  GPS  anomaly  resolution; 

3.  The  formulation  of  the  needed  probabilities  from  the  historical  satellite  data; 
and 

4.  The  generation  of  rules  which  capture  GPS  knowledge  for  preliminary  expert 
system  prototype.  A  discussion  and  summary  of  the  results  is  also  necessary. 

3.2  Applicability  of  the  Approach 

Because  Bayesian  theory  is  based  upon  fundamental  coherent  mathematical 
concepts,  this  thesis  will  determine  if  it  offers  a  sound  basis  upon  which  an  expert 
system  can  be  developed.  This  will  involve  examining  how  the  various  probabil¬ 
ity  and  conditional  relationships  are  developed  and  how  they  interact.  This  will 
also  involve  determing  if  the  system  knowledge  can  be  adequately  expressed  in  a 
probabilistic  manner. 
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Certain  assumptions  are  needed  in  order  for  Bayesian  reasoning  to  be  useful, 
therefore  it  will  be  determined  if  this  particular  type  of  problem  meets  those  criteria 
and  what  are  the  system  limitations. 

3.3  Anomaly  Resolution 

Presently  GPS  anomaly  resolution  is  accomplished  in  two  phases — each  with 
several  steps.  Upon  noticing  anomalous  telemetry,  phase  one  is  for  the  operator  (usu¬ 
ally  not  an  expert)  to  infer  what  class  of  problem  he  potentially  faces  based  upon 
the  anomalous  telemetry.  This  leads  to  a  checklist  which,  written  as  a  flowchart, 
guides  the  operator  to  an  action  which  will  either  remedy  the  problem  or  put  the 
vehicle  in  a  safe  configuration.  If  it  is  necessary  to  “safe”  the  vehicle,  phase  two  is 
for  an  expert  to  review  the  evidence  (telemetry)  prior  to  the  anomaly,  decide  what 
went  wrong,  and  based  upon  the  subjective  utility  of  a  particular  action,  decide  what 
should  be  done.  This  is  a  heuristic  process  which  relies  upon  the  expert’s  ability  to 
reason  with  incomplete  or  uncertain  data  and  reach  a  decision  which  maximizes  the 
utility  for  the  satellite  and  the  user.  For  this  research,  interviews  with  GPS  subsys¬ 
tem  engineers  were  conducted  in  order  to  understand  what  processes  are  involved  in 
making  decisions  which  govern  the  weil-  being  of  the  satellite.  This  helped  define  the 
reasoning  process  that  the  expert  system  should  follow  and  also  in  ascertaining  the 
utility  value  of  selected  actions.  A  review  of  the  vehicle  subsystem  technical  manuals 
as  well  as  contingency  procedures  also  provided  needed  background  knowledge. 

3.4  Data  Generation 

Based  upon  archived  data  of  a  satellite’s  history,  the  actual  prior  probabilities 
needed  in  Bayesian  analysis  were  computed.  Expert  opinion  as  well  as  past  trends 
allowed  accurate  probabilities  of  particular  anomalies  and  their  interrelationships  to 
be  determined.  This  facet  of  the  research  was  crucial  to  the  overall  success  of  the 
project.  Interviews  with  GPS  vehicle  engineers  as  well  as  a  review  of  statistical  data 
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collection  methods  were  required  in  order  to  validate  results. 

3.5  Rule  Base  Development 

A  small  decision-analytic  reasoner,  which  combines  information  in  a  Bayesian 
manner,  was  developed  and  a  basic  example  using  data  obtained  from  GPS  engineers 
was  coded.  This  facilitated  testing  and  validating  the  approach.  It  also  helped 
in  determining  the  utility  of  this  approach  in  representing  system  knowledge  and 
handling  uncertainty  in  expert  systems  of  this  type. 

3.6  Summary 

Throughout  this  thesis  project,  an  effort  was  made  to  bridge  the  gap  between 
decision  theory  and  expert  system  development.  With  satellite  fault  diagnosis  as 
the  chosen  domain  of  application,  this  thesis  hopes  to  show  that  these  areas  can  be 
merged  in  a  practical  and  useful  manner. 
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IV.  Global  Positioning  System  (GPS) 


4-1  Introduction 

The  purpose  of  this  thesis  is  to  determine  if  a  decision  theory  approach  to 
rule-based  expert  systems  is  feasible  and  practical.  The  Global  Positioning  System 
(GPS)  satellite  is  the  system  chosen  to  determine  the  viability  of  the  approach. 
Anomaly  classification  and  present  contingency  procedures  describing  GPS  satellite 
operations  are  reviewed  in  order  to  determine  the  scope  needed  by  the  expert  system. 
Background  knowledge  on  GPS  and  its  mission,  why  this  satellite  program  was 
singled  out.  and  which  subsystem  will  be  examined  for  rule  base  development  is  also 
given. 

4-2  Anomaly  Classifications 

As  defined  in  Section  3.0.2  of  the  GPS  Orbital  Operations  Handbook  (OOH) 
(27),  an  anomaly  is  the  improper  or  unexpected  operation,  condition,  or  behavior  of 
any  portion,  subsystem,  or  system  of  a  space  vehicle.  An  anomaly  may  manifest  itself 
in  an  out-of-limits  parameter  or  adverse  trending  of  any  parameter,  as  indicated  in 
telemetry  data;  failure  of  the  space  vehicle  to  respond  to  commanding;  off-nominal 
attitude  or  orbit  conditions;  or  non-nominal  indications  obtained  from  the  GPS 
Control  Segment  or  Camp  Parks  Communications  Annex.  The  various  levels  of 
anomalies  are  defined  as: 

1.  Level  I  Anomaly.  A  Level  I  anomaly  could  cause  a  total  loss  of  mission  and 
requires  an  immediate  response. 

2.  Level  II  Anomaly.  A  Level  II  anomaly  could  cause  a  total  loss  of  mission  but 
allows  some  time  to  respond. 

3.  Level  III  Anomaly.  A  Level  III  anomaly  could  cause  a  problem  less  severe  than 
a  total  loss  of  mission  and  requires  an  immediate  to  long-term  response. 


4-1 


4-3  Discussion 

The  problems  considered  in  this  thesis  are  Level  II  and  III  anomalies.  The  as¬ 
sessment  of  utility  values  pertaining  to  contingency  actions  concerning  the  anomaly, 
can  be  weighted  in  order  to  examine  problems  of  a  higher  priority  before  examining 
those  of  lower  priority  should  instances  arise  where  evidence  lends  itself  to  different 
problems.  However,  in  the  Orbital  Operations  Handbook  (OOH),  there  is  no  ex¬ 
plicit  utility  scale  ranking  for  anomalies  except  for  the  Level  I,  Level  II,  and  Level 
III  classifications.  Within  each  of  these  levels  there  are  none.  This  knowledge  is 
something  that  is  supposed  to  be  learned  as  the  operator  becomes  more  familiar 
with  the  outcome  of  certain  problems.  Likewise,  recognizing  the  boundary  between 
the  different  classes  starts  to  blur  as  time  is  factored  into  the  problem.  Consequently 
a  Level  III  problem  undetected  can  manifest  itself  as  a  Level  II  anomaly  and  so  forth. 
The  heuristic  experience  that  comes  from  long  term  program  association  is  the  type 
needed  to  be  captured  in  a  expert  system  knowledge  base. 

Presently,  anomalies  are  resolved  using  contingency  checklists  which  have  enu¬ 
merated  most  of  the  foreseeable  problems.  These  checklists  are  written  in  a  posterior 
type  manner — hypotheses  are  concluded  based  on  the  evidence  observed  by  the  op¬ 
erator.  They  are  rigidly  written  and  appear  to  allow  for  no  subjective  interpretation. 
However,  this  aspect  is  aided  by  human  judgement  as  the  operator  gets  more  familiar 
with  the  telemetry  readings.  For  example,  if  the  checklist  requires  that  telemetry 
point  X  =  0,  and  the  user  knows  that  0.1  is  essentially  zero  due  to  the  particu¬ 
lar  traits  of  that  vehicle,  he  considers  this  value  as  zero  and  proceeds.  This  type 
of  subjective  assessment  is  what  is  lacking  in  many  expert  systems  which,  like  the 
OOH  checklists,  are  written  in  a  rigid  form  and  fail  to  allow  for  the  belief  values  to 
be  coherently  incorporated  into  the  knowledge  base.  As  mentioned  previously,  the 
probabilistic  representations  of  uncertainty  by  decision  theory  models  can  be  used 
to  encompass  this  type  of  data  assessment. 
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Presently,  anomaly  resolution  in  all  but  the  simplest  of  cases  usually  requires 
a  group  decision  before  major  corrective  actions  are  initiated.  Due  to  the  nature 
of  most  problems  and  the  inherent  reliability  of  the  satellite,  usually  no  important 
decisions  have  to  be  made  in  such  a  short  time  that  the  situation  cannot  be  throughly 
examined  before  proceeding.  In  situations  like  these,  an  expert  system  can  be  used 
to  augment  the  knowledge  of  the  operations  crew.  Nonetheless,  there  are  problems 
which  do  require  immediate  action,  and  the  operations  crew  must  be  proficient  in 
performing  the  necessary  tasks.  For  situations  like  these,  consultation  with  an  expert 
system  may  be  inappropriate  due  to  the  criticality  of  time. 

4.4  What  is  GPS 

The  Navstar  GPS  is  a  Air  Force  operated  satellite  program  whose  mission  is  to 
provide  highly  precise  position,  velocity,  and  time  information  to  users  around  the 
world  and  to  detect  nuclear  detonations  (NUDETs).  The  GPS  system  is  made  up  of 
four  system  segments:  Space  Segment,  Control  Segment,  Navigation  User  Segment, 
and  NDS  User  Segment. 

4-4-1  Space  Segment.  The  Space  Segment,  when  fully  operational,  will  con¬ 
sist  of  21  Block  II  operational  satellites  and  three  active  spares  placed  in  six  orbital 
planes,  each  having  an  inclination  of  55  degrees.  The  satellites  will  operate  in  circu¬ 
lar  20,200  kilometers  (10,900  nautical  miles)  orbits  with  a  period  of  12  hours  (6:2). 
The  satellites  will  be  spaced  in  such  a  manner  that  a  minimum  of  four  satellites  will 
be  visible  by  any  user  at  any  time,  thereby  ensuring  worldwide  coverage. 

The  GPS  operates  on  two  L-band  frequencies:  1575.42  MHz  (Ll)  and  1227.6 
MHz  (L2).  Each  satellite  is  designed  to  transmit  an  Ll  and  L2  signal.  Ll  carries  a 
precision  (P  code)  signal  and  a  coarse/acquisition  (C/A  code)  signal,  while  L2  carries 
the  P  code  only.  The  use  of  two  frequencies  allows  the  navigation  user  with  a  higher 
accuracy  set  to  adjust  to  the  variable  delay  experienced  by  different  signals  as  they 
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pass  through  the  ionosphere.  Superimposed  on  these  signals  will  be  navigation  data, 
the  satellite's  current  ephemeris  coordinates  (position),  state  of  health  information, 
and  satellite  clock  bias  information. 

In  the  NUDET  detection  role,  each  satellite  utilizes  onboard  sensors  to  detect 
nuclear  events  on  or  above  the  earth’s  surface.  A  satellite  detecting  the  nuclear 
events  processes  and  crosslinks  the  data  to  other  GPS  satellites  in  radio  frequency 
(RF)  proximity  via  UHF  frequencies.  All  satellites  with  the  nuclear  event  data 
transmit  to  the  NDS  User  Segment  via  the  L3  frequency. 

4-4-2  Control  Segment.  The  Control  Segment  includes  a  number  of  monitor 
stations  and  ground  antennas  located  throughout  the  world.  The  monitor  stations 
use  a  GPS  receiver  to  passively  track  all  satellites  in  view  and  thus  accumulate 
ranging  data  from  the  satellite  signals.  The  data  is  also  processed  to  determine 
each  satellite’s  precise  ephemeris  coordinates  and  the  prevailing  time  registered  by 
the  onboard  atomic  clock.  Detected  errors  are  corrected  by  transmitting  new  upload 
data  to  the  satellite  on  a  periodic  basis.  The  state-of-health  of  the  satellite  subsystem 
and  payloads  are  also  monitored. 

4-4-3  Navigation  User  Segment.  The  Navigation  User  Segment  receives  and 
processes  Ll  and  L2-band  navigation  data  radiated  from  the  Space  Segment.  The 
user  passively  receives  coded  signals  from  four  or  more  satellites  in  view  of  his  re¬ 
ceiver.  The  coded  signals  from  three  satellites  are  used  to  triangulate  the  user's 
velocity  and  coordinates.  Data  from  the  fourth  satellite  is  used  to  eliminate  any- 
timing  errors  in  the  user’s  set  clock. 

4-4-4  NDS  User  Segment.  In  support  of  the  Nuclear  Detonation  (NUDET) 
Detection  System  (NDS)  mission,  the  GPS  Block  II  satellite  provides  the  capabil¬ 
ity  to  transmit  Global  Burst  Detector  (GBD)  collected  NUDET  data  to  NDS  User 
Segment  terminals  by  direct  transmission  using  the  L3  downlink  and  through  trans- 
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mission  by  GPS  satellites  within  RF  proximity  via  the  UHF  crosslink.  NDS  users 
process  these  data  together  along  with  navigation  data  to  determine  the  location 
and  charrcteristics  of  the  nuclear  event(s). 

4-5  Applications 

Because  of  the  versatility  and  the  increased  navigational  capability  it  offers. 
GPS  provides  many  benefits  to  both  civilian  and  military  operators.  Knowledge 
of  precise  three-dimensional  position  data  relative  to  friendly  and  enemy  forces'  is 
fundamental  to  the  success  of  a  large  number  of  military  missions,  while  increased 
commercial  traffic  in  the  air  and  on  the  sea  can  utilize  this  information  for  increased 
safety  and  efficiency. 

4-5.1  Military  Applications.  The  substantial  navigation  performance  improve¬ 
ments  afforded  by  the  GPS  can  enhance  many  areas  of  military  operations.  In  air 
operations,  GPS  accuracy  can  streamline  en  route  and  terminal  navigation,  thereby 
reducing  flight  times  and  conserving  fuel.  Because  GPS  is  a  three-dimensional  sys¬ 
tem.  descent  and  landing  operations  can  be  more  closely  controlled.  Combat-related 
applications  involving  bombing  and  ballistic  weapon  delivery  are  also  vastly  im¬ 
proved.  GPS  also  allow  ground  forces  the  capability  of  enhancing  activities  such  as 
site  surveying,  field  artillery  placement,  and  target  acquisition  and  location.  Naval 
forces  also  benefit.  Harbor  entry  operations  can  be  improved  and  activities  such  as 
mine  placement  can  be  made  more  precise  and  safe.  These  are  but  a  few  of  the  many 
military  applications  that  will  benefit  from  GPS  (see  Figure  4.1)  (6:9). 
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Figure  4.1.  Military  Applications 


4-5.2  Civilian  Applications.  The  GPS  will  provide  a  broad  spectrum  of  civil 
users  with  accurate  position,  velocity,  and  time  determination  cabability  at  a  rea¬ 
sonable  cost.  Civil  users  of  air,  sea,  and  ground  vehicles  will  benefit  from  the  GPS 
for  optimal  course  navigation,  which  will  reduce  fuel  costs  and  transportation  time. 
Besides  providing  substantial  benefits  in  air  navigation  and  landing  operations,  air 
search  and  rescue  techniques  will  also  be  greatly  improved.  Mineral  exploration, 
and  accurate  positioning  of  oil  exploration  are  also  some  other  commercial  uses  of 
GPS.  There  are  several  potential  applications  of  GPS  that  will  occur  as  the  system 
becomes  more  accepted  by  the  civilian  community  (see  Figure  4.2). 


Figure  4.2.  Civilian  Applications 


4-6  Space  Vehicle  Subsystems 

GPS  Block  II  satellites  consist  of  a  integrated  assembly  of  nine  subsystems  and 
two  payloads  to  provide  a  space-based  radio  frequency  enviroment  for  the  Global 
Positioning  System. 

The  space  vehicle  consists  of  the  following  subsystems  and  payloads: 


1.  Structure  Subsystem  (STR) 

2.  Thermal  Control  Subsystem  (TCS) 


3.  Electrical  Power  Subsystem  (EPS) 

4.  Attitude  and  Velocity  Control  Subsystem  (AVCS) 

5.  Orbital  Insertion  Subsystem  (OIS) 

6.  Reaction  Control  Subsystem  (RCS) 

7.  Telemetry,  Tracking,  and  Command  Subsystem  (TT&C) 

S.  Navigation  Payload  (NPD) 

9.  L-Band  Subsystem  (LBS) 

10.  Integrated  Transfer  Subsystem  (ITS) 

11.  Global  Burst  Detector  (GBD) 

The  AVCS,  OIS.  and  RCS  subsystems  are  elements  of  the  control  system  (C'S). 
The  ITS  and  GBD  subsystem  are  part  of  the  Nuclear  Detonation  (NUDET)  Detec¬ 
tion  System  (NDS)  payload  (27).  See  Appendix  A  for  a  summary  of  each  subsystem 
as  given  in  the  GPS  Orbital  Operations  Handbook  (OOH)  Section  1.0. 

4.7  Why  GPS? 

The  Global  Positioning  System  was  chosen  as  the  prototype  model  for  ap¬ 
plication  of  the  decision-analytic  approach  to  rule-base  development  for  a  number 
of  reasons.  The  main  reason  is  that  the  Air  Force  is  becoming  more  reliant  upon 
space- based  resources  and  expert  systems  will  play  an  increasing  role  in  managing 
and  maintaining  these  systems.  Without  question,  GPS  is  one  space  resource  which 
figures  to  play  a  very  important  role  in  future  military  operations.  Consequently,  the 
viability  of  this  approach  in  augmenting  decision-making  and  system  control  needs 
to  be  examined  in  the  context  of  its  possible  uses.  Another  strong  point  is  that 
because  the  first  generation  of  GPS  satellites  (Block  I)  have  been  in  an  operational 
test  phase  since  1978,  there  is  a  large  data  base  from  which  system  anomalies  and 
trends  can  be  determined.  This  provides  the  information  needed  to  ascertain  the 
utility  of  certain  actions  as  well  as  form  the  priors  and  likelihood  ratios  needed.  And 
lastly,  because  a  satellite  is  a  complex  system,  the  feasibility  of  the  approach  on  large 
systems  can  be  determined. 
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4-8  System  Examined 

For  this  study,  the  Electrical  Power  Subsystem  concerning  the  loadshed  timers 
will  be  the  chosen  area  in  which  contingency  system  knowledge  will  be  modeled. 
After  conferring  with  Technical  Analysts  from  Lockheed  Missile  and  Space  Corp.. 
the  EPS  system  was  chosen  because  of  its  importance,  and  because  most  of  the  com¬ 
mon  anomalies  occur  in  the  EPS.  Furthermore,  extensive  data  concerning  the  EPS 
subsystem  have  been  kept  which  will  assist  in  determining  the  needed  conditional 
probabilities  of  anomalous  conditions  given  telemetry. 

4-9  Conclusion 

The  purpose  of  this  section  of  the  thesis  was  to  give  background  information 
on  the  GPS  satellite  system  and  the  reasons  it  was  selected  as  the  model.  Anomaly 
classifications  and  contingency  procedures  were  reviewed  as  well  as  the  applicability 
of  satellite  expertise  to  expert  system  technology.  The  Electrical  Power  Subsystem 
was  chosen  as  the  subsystem  in  which  a  knowledge  base  will  be  developed  to  test 
the  viability  of  the  method.  The  next  chapter  discusses  the  assumptions  that  the 
decision-analytic  model  is  be  based  upon. 
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V.  Model  Description 


5.1  Introduction 

This  section  of  the  thesis  discusses  the  decision-analytic  approach  to  developing 
expert  system  rules  which  capture  the  knowledge  of  a  particular  problem  domain. 
The  framework  is  that  of  decision  analysis  encompassing  Bayesian  reasoning  and 
utility  theory.  The  strengths  of  the  methodology  are  that  uncertainty  can  be  incor¬ 
porated  into  the  decision-making  process  in  a  consistent  manner;  the  outputs  can  be 
validated  and  verified;  and,  given  the  initial  model  accurately  describes  the  piobiem 
domain,  the  method  is  fairly  robust.  This  chapter  addresses  the  assumptions  upon 
which  this  system  is  based,  why  a  system  with  its  capabilities  are  needed,  and  the 
major  differences  between  this  rule  base  and  more  traditional  ones.  Morlan’s  (22) 
in-progress  work  was  used  as  a  guideline  for  the  layout  of  the  initial  part  of  the 
chapter. 

5.2  Overview 

Bales'  theorem  is  the  foundation  upon  which  this  approach  to  rule-based  ex¬ 
pert  system  development  is  based.  Should  a  review  of  this  theorem  and  its  basic 
concepts  (i.e.,  priors,  likelihoods,  posteriors)  be  required,  Berger  (1)  or  Morgan  (21) 
are  recommended  references. 

Bayesian  analysis  represents  a  method  for  incorporating  available  relevant  in¬ 
formation  directly  into  the  process  of  making  inferences  about  an  underlying  state 
of  nature,  and  in  formulating  decisions  based  on  these  inferences.  Furthermore,  it 
allows  for  taking  account  of  subjective  belief  in  the  evidence  (second-order  uncer¬ 
tainty)  in  arriving  at  these  inferences.  As  mentioned  in  the  literature  review,  several 
methods  of  incorporating  this  uncertainty  have  been  developed,  yet  few  if  any  do  so 
in  a  coherent  manner  and  fewer  still  use  utility  as  a  measure  of  delineating  between 
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choices.  This  approach  of  managing  knowledge  in  expert,  systems  attempts  to  oper¬ 
ate  in  a  manner  that  more  closely  resembles  rational  decision-making.  In  so  doing, 
the  knowledge  base,  which  has  probabilistically  captured  domain-specific  knowledge, 
coherently  manipulates  a  user’s  belief  about  the  evidence  and  recommends  the  ac¬ 
tion  yielding  the  most  utility.  However,  unlike  conventional  expert  systems,  these 
recommendations  are  made  from  rules  which  are  more  analogous  to  mathematical 
functions  than  "If  . . .  then. . .”  statements.  Whether  this  rational  approach  toward 
uncertainty  management  can  be  sucessfully  mapped  into  human  reasoning,  which  is 
many  times  an  illogical  and  incoherent  process,  remains  to  be  seen. 

5.3  Assumptions 

These  are  the  assumptions  that  are  used  in  the  application  of  this  method: 

1.  Hypotheses  are  assumed  to  be  disjoint  when  the  conditional  probability  and 
conditional  utility  information  is  gathered, 

2.  Second-order  uncertainty  (uncertainty  in  the  evidence  state]  is  independent 
of  the  hypothesis-to-evidence  uncertainty, 

•3.  Actions  are  assumed  to  be  disjoint, 

4.  Evidences  are  not  necessarily  conditionally  independent  within  a  hypothesis 
(i.e.,  P(E1,E2|H)  might  or  might  not  equal  P(E1  |H)*P(E2|H), 

5.  Only  a  relevant  subset  of  the  hypotheses  needs  to  be  considered. 

5.3.1  Disjoint  Hypotheses.  Assuming  disjoint  hypotheses  is  a  fairly  common 
way  to  decompose  problems.  It  allows  for  subjective  reasoning  about  uncertainty 
by  collecting  the  conditional  probability  ( P[E| H] )  of  the  evidences,  E,  conditioned 
on  the  hypotheses,  H  (the  likelihoods).  That  concept  is  also  extended  to  the  utility 
information  by  the  conditional  utility  function,  U [ A | H] . 
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Some  of  the  strong  points  of  this  assumption  are: 


•  It  provides  a  natural  decomposition  of  larger  problems  into  smaller  more  man¬ 
ageable  ones; 

•  It  is  a  logical  partition  for  collecting  evidence-to-cause  probability  relationships; 

•  It  is  a  logical  partition  for  collecting  action-to-cause  utility  information. 

Some  of  the  disadvantages  are: 

•  The  requirement  to  construct  a  separate  hypothesis  that  represents  the  joint 
occurance  of  hypotheses  that  may  occur  together; 

•  The  requirement  to  include  data  on  all  possible  hypotheses.  However,  this  task 
is  required  of  any  type  of  expert  system  that  strives  to  make  rational  decisions. 

5.3.2  Second-Order  Uncertainty.  Second-order  uncertainty  is  uncertainty  con¬ 
cerning  the  state  of  the  evidence.  By  using  an  alternate  form  of  Bayes1  Rule,  some¬ 
times  noted  as  Jeffrey’s  Rule,  uncertainty  can  be  applied  to  the  evidence  in  a  consis¬ 
tent  manner  that  adheres  to  the  laws  of  probability  yet  allows  for  subjective  opinion 
to  be  addressed.  Mathematically,  Jeffrey’s  Rule  is  (7:2) 

P*(4)  =  ]rP(A  | 

1=1 

Though  apparently  sound  in  its  adherence  to  probability  theory,  there  is  still 
some  disagreement  among  probablisticians  as  to  its  validity  or  need.  Pearl  (24)  and 
Snow  (30)  have  professed  differences  to  Jeffrey’s  Rule  and  Morlan  (22)  is  presently 
working  on  a  concept  which  contradicts  with  Jeffrey’s  belief  in  the  relationship  of 
different  evidences.  Nonetheless,  this  thesis  used  Jeffrey’s  Rule  to  demonstrate  the 
concept  of  modifying  the  belief  in  a  hypothesis  given  the  uncertainty  in  the  evidence. 
See  referenced  articles  for  alternate  methods  of  incorporating  additional  information. 
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The  following  examples  demonstrate  a  objective  method  toward  handling  second- 
order  uncertainty  by  utilizing  Jeffrey’s  Rule.  Example  1:  Assume  P[H1|  E2]  =  .7, 

P [H 1  j  -E2]  =  .262.  Now,  suppose  your  belief  in  the  evidence  (E2)  is  sixty  percent 
(i.e.,"P[E2]”  =  .6).  How  can  this  belief  be  coherently  incorporated  in  the  posterior 
of  the  hypothesis  given  the  evidence?  Bayes’  (Jeffrey’s)  Rule  allows  us  a  manner  in 
which  we  can  reassess  this  belief: 

-P[E2]”  -  .6  therefore  “P[-E2]”  =  .4 

P[H\  |  -‘£[£2]’’]  =  P[H\  |  £2]  •  “£[£2]”  +  P[H  1  |  -£2]  •  “£[-£2]” 

=  (,7)(.6)  +  (.262)(.4) 

=  .42  +  .1048 
=  .5248 


Figure  5.1.  Jeffrey’s  Rule  2D  Representation 


Because  of  the  uncertainty  of  the  evidence,  the  belief  in  the  hypothesis  should 
fall  somewhere  between  the  extremes  of  when  you  are  sure  of  seeing  the  evidence 
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P[H1|  E2]  and  when  you  are  sure  you  did  not  see  the  evidence  P[H1|  -E2j.  See 
Figure  5.1  for  graphical  interpretation. 

Example  2:  Assume 

P[Hl  |  £1,£2]  =  .9 

P[Hl  |  £l,-£2]  =  .8 

P{H\  |  -E\,E2)  =  .7 

P[H\  |  -El,-E2]  =  .6 

Suppose:  "P[E  l]”  =  .3  and  “^[£2]”  =  .2 

Therefore:  “P[-£  1]”  =  .7  and  “P[-£2j”  =  .8 

Then:  P[Hl  \  “P[£l]”,  “P[£2]”j  = 

=  (.9)(.3)(.2)  +  (.8)(.3)(.8)  +  (.7)(.7)(.2)  +  (.6)(.7  )(.8) 

=  .054  +  .192  +  .098  +  .336 

=  .68 


Figure  5.2.  Jeffrey’s  Rule  3D  Representation 
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This  value  is  between  P[Hl|El,E2]  =  .9  and  P [H 1  |-E1,-E2]  =  .6  just  as  would  be 
expected.  Figure  5.2  shows  the  three  dimensional  representation.  This  manipulation 
of  uncertainty  will  be  discussed  later  in  the  actual  problem  application. 

5.3.3  Disjoint  Actions.  The  assumption  that  the  actions  are  disjoint  is  simi¬ 
lar  to  the  assumption  that  the  hypotheses  are  disjoint  in  that,  without  it,  the  solution 
space  would  be  combinatorially  large.  Given  N  actions,  a  data  base  allowing  all  com¬ 
binations  would  be  2n.  For  purposes  of  this  thesis,  only  one  action  is  recommended. 
It  is  understood  that  this  might  be  unrealistic  in  terms  of  real  world  situations. 

5.3.4  Conditional  Independence.  Most  Bayesian  problems  assume  evidences 
are  conditionally  independent.  In  such  a  case,  the  joint  probabilities  are  simply  the 
individual  probabilities  multiplied  together  (i.e.,  P[El,E2  [Hi]  =  P[El|Hl]*P[E2  ] H 1  ] ) 
Many  times  this  is  indeed  a  valid  assumption  which  helps  in  combining  several  pieces 
of  information.  Snow  (30:635)  points  out  several  ways  in  which  data  can  be  manip¬ 
ulated  into  satisfying  this  simplifying  assumption.  For  one,  the  evidences  may  be 
restated  or  algebraically  transformed  so  that  they  are  conditionally  independent  or 
it  may  be  possible  to  combine  dependent  evidences  into  one  evidence  which  can  be 
considered  independent  across  the  hypothesis  set.  All  in  all,  “Performing  statisti¬ 
cal  inference  under  the  conditional  independence  restriction  is  a  lot  like  modeling 
physical  systems  using  linear  relationships.  In  both  cases,  the  restrictive  assumption 
makes  the  output  a  simple  combination  of  the  responses  for  each  separate  input.” 
(30:635)  Nevertheless,  in  many  real  world  situations  there  are  times  when  this  as¬ 
sumption  cannot  be  made.  When  situations  are  encountered  where  evidences  are 
dependent  given  an  hypothesis,  their  relationships  must  be  accurately  captured  in 
the  knowledge  base  if  the  expert  system  is  going  to  provide  realistic  assistance.  The 
model  does  allow  for  the  accurate  representation  and  manipulation  of  dependent 
data. 
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5.3.5  Plausible  Set.  Given  a  certain  problem  domain  (subset  of  large  system), 
heuristics  are  used  to  narrow  down  a  set  of  expected  hypotheses  which  pertain  to 
the  domain  in  question.  If  this  assumption  is  not  made,  the  database  would  become 
too  large  trying  to  enumerate  every  possible  (no  matter  how  unlikely)  state  of  nature 
(hypothesis). 

This  "pruning”  of  the  hypothesis  space,  which  is  a  subjective  process  in  itself, 
would  involve  determining  which  hypotheses  are  deemed  most  likely  to  occur.  As  in 
traditional  rule-based  systems,  this  “plausible”  set  of  hypotheses  is  based  upon  past 
history,  similar  problems,  or  educated  guesses.  However,  unlike  traditional  rule-based 
systems,  the  utility  of  not  considering  certain  hypotheses  must  also  be  factored  into 
the  decision  of  determining  the  plausible  hypothesis  set.  Some  threshold  level  must 
be  decided  upon  that  takes  into  consideration  the  likelihood  of  the  hypothesis  and 
also  the  expected  loss  should  that  hypothesis  not  be  considered.  By  examining  utility 
information,  hypotheses  which  might  be  excluded  because  of  their  remoteness  might 
enter  into  the  plausible  set  because  they  are  just  too  “expensive”  to  not  account  for 
their  happening. 

0.4  Justification 

As  mentioned  previously  in  the  literature  review,  there  are  several  reasons  why 
decision  theory  is  a  desirable  approach  in  expert  system  development.  The  primary 
reason  is  the  ability  to  coherently  handle  uncertainty.  The  following  paragraphs  ex¬ 
amine  the  fundamental  differences  between  this  type  of  system  and  more  traditional 
ones,  and  why  this  approach  appears  more  suitable  to  real  world  applications. 

As  with  any  knowledge  base  that  attempts  to  model  complex  systems,  a  defini¬ 
tive  approach  must  be  used  in  the  formulation  of  rules.  Generally,  there  are  two  ways, 
one  which  does  not  incorporate  uncertainty  or  implication  and  one  which  does;  the 
former  being  superficial  yet  functional  in  ideal  situations,  the  latter  being  a  great 
deal  more  complex  yet  incorporating  a  more  realistic  attitude  toward  quantifying 
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knowledge. 

The  first  approach  merely  looks  at  indicators  which  are  believed  to  be  symp¬ 
toms  of  suspected  problems.  Much  like  the  belief  that  a  fever  is  a  symptom  of  some 
type  of  condition  (e.g.,  measles,  flu,  etc.).  This  approach  accepts  the  indication  (evi¬ 
dence)  of  the  fever  as  being  true  and  then  proceeds  to  determine  what  would  cause  it 
and  subsequently  how  to  treat  the  condition.  A  rule  might  be  “If  patient  has  a  fever 
of  >  99.3,  then  prescribe  aspirin.”  No  allowance  is  given  for  subjective  assessment 
considering  external  situations  which  might  effect  the  strength  of  the  implication  or 
the  user’s  belief.  Most  rule-based  systems  are  one  dimensional  in  this  respect  and 
operate  accordingly,  many,  it  must  be  noted,  with  quite  satisfactory  results. 

The  other  approach  is  more  complex,  looking  at  each  piece  of  evidence  in  the 
context  of  the  entire  situation.  Given  the  same  situation  of  a  person  having  a  fever, 
this  approach  would  allow  subjective  questioning  of  the  evidence  (i.e.,  how  accurate 
is  the  thermometer?  or  who  took  the  reading;  was  it  the  doctor  (expert),  or  a  new 
candy  striper?).  Given  a  rule  base  which  is  formulated  in  this  manner,  each  piece  of 
evidence  is  able  to  be  judged  as  to  its  veracity  and  this  interpretation  can  be  used 
to  determine  the  fault  (anomaly)  and  subsequent  action.  Consequently,  while  the 
implied  relationship  of  99.3  degrees  and  fever  is  stored  in  the  data  base,  your  belief 
(uncertainty)  might  waiver  anywhere  from  1.0  (definite  fever)  to  .2  (most  likely  not 
a  fever)  to  0.0  definitely  not  a  fever,  depending  on  your  assessment  of  the  situation. 
This  degree  of  belief  could  cause  your  next  action  to  range  from  prescribing  aspirin 
to  sending  the  patient  home.  The  advantages  of  a  system  which  can  reason  in  such 
a  manner  are  many,  however  there  are  disadvantages;  one  being  that  the  inference 
engine  needed  is  quite  complex  and  two,  the  size  of  the  rule  base  is  quite  large. 
Nevertheless  this  is  to  be  expected  from  any  system  which  tries  to  approach  a  human 
level  of  reasoning  capability. 

This  decision-analytic  model,  unlike  alternative  rule-based  methods  for  knowl¬ 
edge  representation  and  handling  uncertainty,  does  allow  for  the  incorporation  of 


other  information  in  a  manner  which  adheres  with  the  expert’s  opinion  yet  consis¬ 
tently  propagates  this  belief  throughout  the  system.  Furthermore,  the  strength  of 
the  implication  within  individual  rules  can  be  controlled  by  the  likelihood  data. 

5.5  How  the  Rule  Base  Differs 

The  rules  which  operate  in  this  expert  system  environment  are  not  of  the 
context  ’“if  A  then  B”.  They  function  in  the  same  manner,  yet  are  not  as  simple 
in  structure  nor  form.  Instead  they  operate  more  along  the  lines  of  mathematical 
functions  than  simple  logical  statements.  This  allows  the  rule  base  to  be  more 
dynamic  and  flexible  in  its  manipulation  of  data. 

Stored  probabilistically  in  the  system  is  the  basic  framework  from  which  an 
expert  has  determined  which  situations  warrant  what  actions,  and  what  evidence  can 
be  used  to  determine  the  needed  action.  This  knowledge  is  expressed  in  the  prior, 
likelihood,  and  utility  data.  Using  a  decision  theory  approach,  recommended  actions 
are  based  on  the  highest  utility.  These  recommendations  can  vary  according  to  the 
P(H  |  E)  *  (Da)-  Consequently,  varying  the  probabilities  concerning  the  evidence 
about  the  state  of  the  world  may  lead  to  different  actions.  As  seen  in  Figure  5.3 
below,  various  values  of  E\  and  E?  would  cause  different  courses  of  action  to  be 
pursued.  Because  of  this  functional  relationship,  the  rules  are  not  all  enumerated 
beforehand,  but  merely  the  expert’s  functional  belief  as  to  how  the  information 
sh-  uld  relate  together. 

Comparing  this  output  with  traditional  rule  bases,  it  can  be  shown  that  these 
rules  provide  all  the  capability  of  those  written  in  the  rigid  “If  A  then  B”  format,  yet 
offer  much  more  latitude  in  fault  diagnosis  because  second-order  uncertainty  can  be 
incorporated.  Needless  to  say,  this  is  precisely  the  type  of  system  needed  in  an  area 
such  as  satellite  operations,  where  human  judgment  must  temper  the  information 
seen  in  the  context  of  the  environment. 
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■5.6  Knowledge  Base 

The  knowledge  base  is  made  up  of  three  parts:  the  prior  distribution  of  the 
plausible  hypothesis  set,  the  joint  likelihood  matrix,  and  the  utility  table  values. 
Because  the  joint  likelihood  matrix  represents  the  bulk  of  the  information  of  the 
knowledge  base,  and  yet  differs  largely  in  comparison  with  alternative  rule-based  data 
bases,  its  make  up  and  use  will  be  discussed.  Particularly  how  various  combinations 
of  the  data  are  represented  given  both  independent  and  dependent  evidences. 

Given  the  individual  likelihood  values  provided  by  the  expert,  three  joint  ma¬ 
trices  are  created  representing  different  versions  of  likelihood  information  (see  Fig¬ 
ure  5.4).  These  are  denoted  as  the  P(E|H),  the  P(-E|H),  and  P({E}|H).  These 
values  are  used  to  calculate  the  posterior  distribution  given  the  observance  of  single 
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or  multiple  evidences.  Any  of  the  commercial  spreadsheet  programs  can  facilitate 
the  development  of  these  matrices  due  to  the  rapid  updating  and  mathematical  ca- 
pabilit.es  (this  thesis  used  Quattro™). 


P(llH) 

n 

-III) 

01 

02 

03 

k4 

kl 

k2 

k3 

k4 

pi ,234 

0.3 

0 

0.257 

0.237 

pi, 234 

0.7 

1 

0.743 

0  743 

p2, 134 

0.0 

0 

0.771 

0.771 

p2, 134 

0.1 

1 

0.229 

0.229 

p3, 124 

0.9 

0 

0.1 

0.9 

p3, 124 

0.4 

1 

0.9 

0.1 

p4,123 

0.03 

0.03 

0.05 

0.05 

P4.123 

0.93 

0.95 

0.93 

0.93 

p!2,34 

0.27 

0 

0.194  ( 

1.194147 

pl2,34 

0.07 

1 

0.170  0 

.170147 

pi  3 , 24 

0.23 

0 

0.025 

0.2313 

p  13 , 24 

0.35 

1 

0  949 

0  0743 

pl4,23 

0.013 

0 

0.012 

0.01243 

pl4 ,23 

0.945 

0.95 

6  703 

0  70393 

P23 , 14 

0.54 

0 

0.077 

0.9939 

p23, 14 

0.04 

1 

0.206 

0  0229 

p24, 13 

0  045 

0 

0.039 

0.03933 

p24 , 1 3 

0.093 

9.93 

0.217 

0.21733 

p34, 12 

0.03 

0 

0.003 

0.043 

p34, 12 

9.34 

0.93 

0  955 

0.099 

p  1 23 , 4 

0  223 

0 

0.019  ( 

1.174332 

p 1 23 , 4 

0.033 

1 

0.133  0 

.917014 

p  1 2  4 , 3 

0.013 

0 

0.009  ( 

1.009907 

p 1 24 , 3 

0.044 

0  93 

0.161  0 

.161939 

p 134 , 2 

c  or. 

0 

0.001  ( 

1.011343 

pi  34, 2 

4  332 

0.93 

0  633  0 

.070393 

p 234,1 

0.027 

0 

0.003  ( 

1.034493 

p234,l 

0.039 

0.93 

0.193  0 

.021753 

pi  134 

0.011 

0 

0.000  0.009914 

pi  234 

0.033 

0.99 

0.143  0 

.0161(3 

N(«) 

11) 

k)  12 

k3 

k4 

0 ,1234  0.033  0.99  0.143  0.019163 

pi,  234  0.00473  0  0.030319  0.003991 

p2, 134  0.29923  0  0.499799  0.034421 

pj,  124  0.03323  0  0.919163  0.143473 

p4,123  0.00173  0.03  0.007999  0.000939 

pl2,34  0.04273  0  0.199413  0.019923 

pl3,24  0.02373  0  0.003391  0.030319 

p!4,23  0.00023  0  0.002649  0.009294 

p23 , 14  0  29923  0  0.094421  0.499799 

p24, 13  0.01973  0  0.029779  0.002964 

p34,12  0.00173  0  0.000690  0.007999 

pl23,4  0.21373  0  0.019923  9.196419 

pl24,3  0.00223  0  0.009919  0.000990 

p!34,2  0.00123  0  0.000294  0.002649 

p234,l  0.01973  0  0.002994  0.029779 

p!234,()  0.01123  0  0.000990  0.009919 


Figure  5.4.  Likelihood  Matrices 


The  P(E|H)  and  P(-E|H)  matrices  only  concern  evidence  that  is  observed  to 
be  true/false  and  does  not  take  into  consideration  evidence  not  observed  (i.e..  in  a 
situation  with  four  evidences,  P(1|H)  only  looks  at  the  probability  of  Evidence  1 
being  true  without  considering  the  states  of  Evidences  2,  3,  or  4).  Likewise,  P(-1|H) 
is  Evidence  1  being  observed  as  false  given  a  particular  hypothesis,  without  concern 
for  Evidences  2,3,  or  4.  The  P ( { E }  | H )  matrix  looks  at  all  the  evidence  whether  it  be 
true  or  false  (i.e.,  P ( { 1 ,234 } | H )  would  be  interpreted  as  the  probability  of  Evidence 
1  being  true  and  Evidences  2,  3,  and  4  being  false).  The  P({E}|H)  matrix  is  made 
up  of  the  data  from  the  P(E|H)  and  the  P(-E|H)  matrices.  Because  any  combination 
of  evidence  can  be  derived  from  the  P({E}|H)  matrix,  it  is  the  knowledge  base  that 
the  inference  engine  of  the  expert  system  acts  upon  when  determining  posterior 
probabilities. 

5.6.1  Combining  Evidence.  The  ability  to  incorporate  various  combinations 
of  the  evidence  and  make  rational  decisions  is  an  important  factor  in  knowledge 
reasoning.  The  decision-analytic  model  is  able  to  represent  combinations  of  both 
independent  and  dependent  evidence  in  a  manner  that  accurately  reflects  the  expert's 
knowledge. 


5.6.1. 1  Independent  Evidence.  In  cases  where  the  evidence  is  indepen¬ 
dent  concerning  a  certain  hypothesis,  the  individual  likelihoods  are  multiplied  to¬ 
gether  to  form  joint  probabilities.  Therefore,  the  probability  of  P(E1,E2|H)  is  the 
P(  El  | H )  x  P(E2|H)  when  El  and  E2  are  conditionally  independent.  This  assumption 
makes  it  easy  to  determine  likelihoods  involving  many  different  evidences.  When  El 
and  E2  are  not  conditionally  independent  (see  Section  5. 6. 1.2,  Dependent  Evidence) 
given  the  hypothesis,  those  values  are  hardwired  in  the  system  and  are  used  whenever 
Evidences  1  and  2  are  combined  together.  Under  the  assumption  that  most  evidence 
is  independent  or  nearly  enough  so  for  practical  purposes,  only  a  small  subset  of 
data  needs  to  be  hardwired. 
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5.6. 1.2  Dependent  Evidence.  Because  it  is  unrealistic  to  assume  that 
all  of  the  evidence  is  independent,  a  hypothetical  example  was  generated  in  which 
two  pieces  of  evidence  are  dependent  given  a  certain  hypothesis.  This  demonstrates 
how  the  system  handles  dependent  evidence  data. 

Assume  that  in  this  example  the  cause  of  an  automobile  problem  is  the  battery 
being  low  [Hi),  and  evidences  Ex  (headlights  work)  and  £3  (car  turns  over)  are 
dependent.  Initial  likelihoods  are  P[£j  |  H\]  =  .3  and  P[£3  |  Hx]  =  .6.  Assume  the 
joint  occurence  of  both  evidences  is  believed  to  be  .25  (i.e.,  P[£i,  £3  |  £1]  =  .25).  If 
the  evidences  were  treated  as  independent  events,  the  joint  probability  would  be  .18 
( .3  x  .6  =  .18).  This  28%  (.25  -  .18/. 25  )  difference  could  have  significant  ramifications 
involving  decisions  based  on  the  joint  occurence  of  these  evidences.  The  tree  diagram 
below  (Figure  5.5)  graphically  shows  which  numbers  need  to  be  accessed  anytime 
these  two  evidences  are  combined  concerning  Hx. 


Figure  5.5.  Tree  Diagram 


In  this  diagram  the  initial  conditional  probabilities  remain  the  same.  Conse¬ 
quently,  if  the  headlights  do  work,  P(El|Hl]  =  .3,  and  if  they  don’t, P[-El|Hl]  =  .7. 
If  the  car  turns  over,  P[E3|Hl]  =  .6,  and  if  it  doesn’t,  P[-E3|Hl]  =  .4.  However, 
the  dependency  of  the  evidences  requires  four  additional  conditional  probabilities  to 
be  saved  in  the  system  representing  joint  relationships.  These  probabilities  are  as 
follows: 

P[El,E3|  HI]  =  .25 
P[E1,-E3|  HI]  =  .05 
P[-E1,E3|  Hi]  =  .35 
P[-E1,-E3|  HI]  =  .35 

The  numbers  used  are  hypothetical  and  are  used  merely  to  demonstrate  the 
necessity  of  accurately  capturing  the  joint  likelihoods  of  dependent  evidences.  Con¬ 
sequently,  any  time  El  and  E3  are  used  concerning  Hi,  these  numbers  need  to  be 
used.  As  for  other  hypotheses,  the  probabilities  are  merely  combined  as  any  other 
independent  piece  of  information. 

5.7  Summary 

This  chapter  has  reviewed  the  basic  outline  of  the  decision-analytic  approach 
used  in  this  thesis.  The  theory  behind  Bayesian  reasoning  was  reviewed  and  how  it 
combined  with  utility  information  is  a  rational  decision  making  tool.  The  assump¬ 
tions  used  in  this  method  have  been  reviewed  as  well  how  this  type  of  expert  system 
offers  advantages  over  traditional  rule-based  systems  because  of  its  added  versatil¬ 
ity.  Why  the  rules  in  this  model  operate  and  appear  like  mathematical  functions 
was  discussed  as  well  as  how  the  knowledge  base  is  developed.  The  next  chapter 
discusses  model  applications. 


5-14 


VI.  Generic  Formulation 


6.1  Introduction 

The  goal  of  expert  systems  is  to  be  able  to  accurately  represent  and  utilize 
knowledge  in  a  manner  that  closely  resembles  human  intelligence.  As  debated  in  the 
literature  review,  the  ability  to  capture  and  reason  with  uncertainty  is  one  aspect  of 
human  intelligence  that  is  particularly  difficult  to  grasp.  Another  heavily  debated 
topic  is  how  should  this  knowledge  be  represented.  Accepting  the  statements  that 
(1)  probability  is  the  only  manner  to  coherently  model  uncertainty  and  (2)  rational 
decision  making  requires  maximizing  utility,  as  being  true,  this  chapter,  under  the 
assumptions  of  Chapter  IV,  sheds  some  light  on  how  information  can  be  probabilis¬ 
tically  captured  in  a  knowledge  base,  and  how  uncertainty  assessments  by  the  user 
are  coherently  manipulated  by  a  decision-analytic  reasoner.  As  with  any  expert  sys¬ 
tem,  the  effectiveness  and  accuracy  of  the  output  is  directly  related  to  the  knowledge 
captured  in  the  data  base.  Consequently,  care  should  be  taken  to  avoid  incorrect 
assumptions,  or  faulty  implications  concerning  the  system  description. 

6.2  Overview 

The  ability  to  accurately  represent  an  expert’s  interpretation  of  how  knowledge 
relates  together  is  an  important  facet  of  expert  system  technology.  Several  promi¬ 
nent  papers  have  been  written  concerning  how  this  process  can  be  done  in  a  manner 
that  is  both  easy  to  apply  yet  quantitatively  and  qualitatively  accurate.  Decision 
trees,  graphs,  or  networks,  including  belief  networks  and  influence  diagrams,  are  well 
known  representations  which  can  facilitate  the  assessment  of  coherent  prior  distribu¬ 
tions  and  data  dependencies.  This  thesis  did  not  have  to  resort  to  influence  diagrams 
and  the  like  because  the  expert  was  able  to  accurately  quantify  his  knowledge  in  a 
probabilistic  manner  without  such  tools.  However,  the  domain  size  in  the  example 
was  small  and  no  doubt  attributed  to  this  result.  Given  a  larger  domained  problem. 
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graphical  representations  such  as  belief  networks  and  influence  diagrams  are  recom¬ 
mended  because  of  their  ease  of  use  and  versatility.  Current  work  by  Morlan  (22) 
and  Deakin  (5)  shows  how  these  knowledge  representation  schemes  can  directly  be 
applied  to  decision-theoretic  models  like  this  one. 

Concerning  decision-theoretic  inference,  the  assessment  of  priors,  likelihoods, 
and  utility  values  are  important  problems. 

Priors,  in  reference  to  the  hypothesis  set,  give  the  initial  probabilities  of  each 
hypothesis  without  the  acquisition  of  additional  evidence.  They  should  reflect  the 
[decision  maker’s]  prior  information  about  the  uncertain  quantity  in  question.  If  this 
information  is  primarily  in  the  form  of  sample  results,  the  priors  should  be  close  to 
the  observed  relative  frequencies.  Under  the  assumption  that  the  hypothesis  set  is 
exclusive  and  exhaustive,  the  sum  of  the  individual  priors  must  equal  one. 

Likelihoods  are  conditional  probabilities  which  capture  the  probabilistic  rela¬ 
tionship  between  evidence  and  hypotheses.  They  denote  the  conditional  probability 
of  obtaining  a  particular  sample  result  (evidence)  given  some  value  of  the  basic 
random  variable  (hypothesis).  They,  as  priors,  should  be  based  on  prior  data,  if 
possible. 

Utility  involves  quantifying  how  useful  or  valued  some  action  or  consequence 
is.  In  the  assessment  of  the  utility  functions,  as  in  the  assessment  of  probabilities, 
an  element  of  vagueness  is  involved.  However,  unless  the  decision-making  problem 
is  highly  sensitive  to  slight  changes  in  the  utility  function,  it  appears  as  if  the  model 
is  fairly  robust  over  a  range  of  utility  values.  (Data  in  this  thesis  support  this 
assumption.) 

6.3  Converting  Knowledge  into  Decision  Analytic  Format 

The  following  outline  shows  how  GPS  contingency  procedures  can  be  fashioned 
into  the  necessary  probabilistic  format.  The  method  should  be  applicable  to  any  type 
of  knowledge  that  can  be  written  in  the  “If  A  then  B”  structure.  The  procedure 


involves  taking  data,  which  in  the  case  of  checklists  is  written  in  a  posterior-type 
format,  and  transforming  them  into  likelihood  ratios  and  plausible  hypotheses.  The 
following  example  was  taken  from  GPS  OOH  3.7-51. 

Data:  Disable  the  loadshed  when  any  of  the  following  conditions  are  met. 

3.7.9. 1.2  LS1TM  and/or  LS2TM  >  4.20  v  during  earth  or  lunar  eclipse. 

3.7.9. 1.3  LS1TM  and/or  LS2TM  >  4.20  v  and  IGS  is  active. 

Step  1)  Determine  state  of  the  world. 

-  E0  LS1TM  >  4.2 

Step  2)  Determine  hypotheses  which  would  explain  present  state  of  the  world. 

Hi  -  Combined  eclipse  (earth  or  lunar). 

H2  -  IGS  is  active. 

Step  3)  Determine  relevant  evidence  (information)  needed  to  discern  between  hypotheses. 

El  -  Vehicle  is  in  eclipse  (earth  or  lunar).  (This  could  be  broken  down  into 
separate  telemetry  values  or  can  be  looked  at  as  just  one  evidence  state  incorporating 
multiple  telemetry  values.) 

E2  -  L3RFP  is  between  12.6  and  22.5. 

Step  4)  Determine  appropriate  response  actions  for  each  hypothesis. 

A1  -  Disable  loadshed. 

After  putting  information  into  the  necessary  format,  subjective  calculations 
of  likelihoods,  priors,  and  utility  actions  are  made.  This  would  include  the  joint 
likelihood  matrix  and  the  action  diagram.  Below  are  hypothetical  assessments. 

•  P(E1|H1,E0)  =  1.00 

•  P(E2|H1,E0)  =  .00001 

•  P  ( E 1 1 H2,  E0 )  =  .0001 


•  P(E2|H2.E0)  =  1.00 

•  P(H1)  =  .990 

•  P(H2)  =  .010 


The  data  is  now  in  the  necessary  format  to  be  accessed  by  a  Decision-Analytic 
Reasoner  (see  Appendix  C,  Scheme  Program).  Upon  the  observation  of  the  LSlTM 
and/or  LS2TM  being  greater  than  4.2V,  this  area  of  the  knowledge  base  would 
be  accessed.  In  this  case,  there  are  only  two  relevant  evidences  which  need  to  be 
observed.  Upon  the  observation  of  these  telemetry  points,  the  appropriate  action 
would  be  recommended.  In  this  case,  the  only  action  is  to  disable  the  loadshed  timer. 
This  is  a  simple  example  which  demonstrates  how  ”If-  •  -then-  •  •"  information  can  be 
captured  probabilistically  in  the  knowledge  base. 

6.4  Likelihood  Determination 

In  order  to  determine  the  appropriate  likelihood  values  to  be  used  in  determin¬ 
ing  the  posterior  probabilities,  the  P({E}  |  H)  table  is  referenced  (see  Figure  6.1). 
This  table  contains  the  set  of  joint  likelihood  values  which  were  determined  by  the 
expert.  For  example,  assume  Evidences  1,  2,  and  3  were  observed,  with  E12  being 
true  and  E3  being  false.  The  table  is  arranged  so  that  all  affirmative  evidences  are 
on  the  left  and  all  negative  responses  to  the  evidence  are  on  the  right,  separated  by 
a  comma  (e.g.,  P(E12,3|H,)  ). 
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P((I)1H) 

hi 

h2  h3  b4 

0,1234 

0.033 

0.95  0.145  0.010103 

pl ,234 

0.00473 

0  0.050319  0.005591 

p2,134 

0.29925 

0  0.489789  0.054421 

P3.124 

0.03325 

0  0.010103  0.143475 

P4.123 

0.00173 

0.05  0.007050  0.000830 

P12.34 

0.04275 

0  0.100413  0.018823 

P13.24 

0.02373 

0  0.003391  0.030319 

pU,23 

0.00023 

0  0.002848  0.000294 

p23, 14 

0.29923 

0  0.054421  0.489789 

p24, 13 

0.01373 

0  0.023778  0.002804 

P34.12 

0.00173 

0  0.000850  0.007030 

P123.4 

0.21375 

0  0.018823  0.109413 

P124.3 

0.00223 

0  0.008910  0.000990 

p!34,2 

0.00123 

0  0.000294  0.002048 

P234.1 

0.01373 

0  0.002804  0.023778 

pl234,  ()  0.01123 

0  0.000990  0.008918 

Figure  6.1.  Joint  Likelihood  Matrix 


However,  the  data  base  table  is  formulated  with  all  evidences  being  observed 
given  a  particular  scenario,  and,  in  this  case,  we  have  only  observed  three  of  the 
possible  four.  In  order  to  determine  the  necessary  likelihoods  considering  only  the 
three  observed  evidences,  all  combinations  of  the  data  where  E12  are  on  the  left 
and  E3  is  on  the  right  are  summed  together.  This  would  involve  the  following 
combinations  of  data:  (El24,3)  and  (El2,34)  (see  Table  6.1.  Assuming  there  is  no 
uncertainty  in  the  evidence  states,  the  likelihood  value  would  be: 

Combined  with  the  prior  probabilities,  these  values  would  be  used  in  determining 
the  posterior  distribution. 
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Table  6.1.  Determining  Likelihood  With  No  Uncertainty 


HI 

H2 

H3 

H4 

(12,34) 

+ 

0.04275 

0 

0.169415 

0.018823 

(124,3) 

0.00225 

0 

0.008916 

0.000990 

(12,3) 

0.045 

0 

0.178331 

0.019813 

6.5  Uncertainty  Assessment 

In  order  to  modify  the  evidence  according  to  the  subjective  belief  assessment, 
a  derivation  of  Bayes’  Rule  (see  Chapter  IV)  is  used.  A  vector  is  created  which 
enumerates  the  various  combinations  of  the  data  incorporating  the  belief  assessments. 
In  cases  where  the  belief  is  some  number  other  than  one,  both  cases  of  confirmation 
and  disconfirmation  are  factored  into  the  assessment.  Consequently,  if  belief  in  some 
evidence  is  .7,  then  the  disbelief  in  that  evidence  must  be  .3.  Table  6.2  below  shows 
the  vector  created  when  the  belief  in  Evidences  1  and  2  are  .9  and  .1,  respectively. 
This  value  is  the  likelihood  value  used  in  determining  the  posterior  probabilities. 

P(  E12,3|  Hi) 


Table  6.2.  Determining  Likelihood  With  Uncertainty 


Vector 

Posterior  (Vector) 

El  E2  E3 

Y(.9)  Y(.l)  N(1.0) 

Y(.9)  N(l-.l)  N(1.0) 

N(l-.9)  Y(.l)  N(1.0) 

N(l-.9)  N(l-.l)  N(1.0) 

P(Hi|El2,3)*P(.9*.l*1.0) 

+  P(Hi|El,23)*P(.9*.9*1.0) 

+  P(Hi|E2,13)*P(.l*.  1*1.0) 

+  P(Hi|E,123)*P(.l*.9*1.0) 

6.6  Posterior  Assessment 

Using  Bayes’  Rule,  posteriors  are  determined  based  upon  the  evidence  observed 
and  the  degree  of  belief.  Posterior  values  involving  each  the  possible  hypotheses 


are  calculated  and  these  values  are  combined  with  the  utility  values  of  the  action 
matrix.  The  DA  reasoner  does  a  comparison  of  the  resultant  utilities  and  using  the 
rational  approach  of  maximizing  utility,  recommends  the  action  yielding  the  highest 
result.  See  Figure  6.2  below  showing  a  2  hypothesis  and  2  action  decision  matrix. 
In  this  case  the  selection  is  made  according  to  the  following  formula  for  expected 
utility: 

If  P(Hl|E)a  +  P(H2|E)b  >  P(Hl|E)c  +  P(H2|E)d 
Then  A1  is  preferred  to  A2. 


HI 

H2 

A1 

a 

b 

A2 

c 

d 

Figure  6.2.  Action  Utility  Table 


6.7  .  tction  Vectors  and  Utility  Assessments 

The  action  matrix  ties  together  all  actions  which  have  been  identified  as  pro¬ 
ducing  definite  benefits  (or  losses)  if  taken  when  the  hypothesis  is  true. 

Given  the  plausible  set  of  hypotheses,  which  can  explain  the  present  state  of 
the  world,  actions  are  determined  which  would  provide  the  most  utility  given  perfect 
information.  In  so  doing,  there  are  as  many  optimal  actions  as  there  are  hypotheses. 
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This  assumes  that  each  hypothesis  would  have  a  different  optimal  action  associated 
with  it. 

Using  a  reference  lottery,  and  a  range  from  0-100,  the  various  actions  of  the 
plausible  set  are  scaled  in  relation  to  one  another  as  to  the  amount  of  utility  they 
provide  given  each  hypothesis.  It  is  assumed  that  the  optimal  action  for  each  hypoth¬ 
esis  is  given  a  value  of  100.  This  represents  doing  the  best  that  can  be  done  under 
the  present  circumstances.  A  value  of  zero  conversely  would  represent  an  action  that 
not  only  did  not  help  the  situation,  but  could  also  be  potentially  damaging.  As 
mentioned  before,  this  ranking  within  each  hypothesis  is  also  a  subjective  process. 
Raiffa  (25),  gives  a  good  description  of  how  these  values  can  be  determined  using  a 
reference  lottery. 

6.8  Determining  Utility  Assessments  Between  Hypotheses 

After  utility  data  is  determined  within  a  particular  hypothesis  for  the  various 
actions,  there  must  be  some  way  of  relating  these  assessments  together  so  that  they 
adhere  to  the  expert’s  overall  belief  in  the  hypothesis’  importance.  This  thesis  used 
a  ratio  formula  based  on  criticality  to  determine  relative  weights.  This  method, 
described  below,  is  by  no  means  the  only  way  to  determine  this  scale  ranking. 

Given  the  plausible  set  of  hypotheses,  the  expert  needs  to  rank  order  them 
from  the  most  critical  to  the  least.  The  least  critical  hypothesis  is  assigned  a  value 
of  1  and  then  the  values  of  the  others  are  determine  by  means  of  ratios.  In  the 
example  used  in  the  thesis  concerning  the  loadshed  1  timer  (see  Chapter  VII),  the 
hypotheses  were  ranked  as  follows: 

1.  Attitude  problem, 

2.  Solar  array  not  tracking, 

3.  Faulty  timer, 

4.  Eclipse  operations. 
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The  eclipse,  which  is  a  routine  and  expected  event,  was  assigned  a  value  of  1 
and  the  the  expert  was  queried  as  to  how  many  times  she  would  be  willing  to  make 
a  wrong  assessment  concerning  the  eclipse  versus  an  attitude  problem.  The  value 
was  100  times  for  the  attitude  problem,  20  times  for  a  bad  timer,  and  50  times  for 
a  solar  array  tracking  problem.  Figure  6.3  shows  the  unsealed  utility  values  initially 
provided  by  the  expert  and  the  scaled  values  which  incorporate  the  ratios.  The 
scaled  utility  table  is  the  one  captured  in  the  knowledge  base  and  one  used  by  the 
DA  reasoner. 


HI  H2  H3  H4  HI  H2  H3  H4 
A1 

A  2 

A3 

A  4 

Unsealed  Scaled 


100 

2 

50 

10 

0 

100 

0 

10 

40 

10 

100 

60 

10 

40 

10 

100 

100 

200 

1000 

500 

0 

10000 

0 

500 

40 

1000 

2000 

3000 

10 

4000 

200 

5000 

Figure  6.3.  Utility  Table 

The  assessment  of  utility,  like  most  probabilistic  inferences,  is  a  subjective 
process.  As  with  probability  assessments,  much  has  been  done  in  the  area  of  de- 
temining  how  to  accurately  capture  an  expert’s  value  judgements  concerning  utility 
assessments.  Studies  show  that  unless  the  decision  making  problem  is  highly  sensi¬ 
tive  to  slight  changes  in  the  utility  factors,  it  shouldn’t  be  too  difficult  to  determine 
a  set  of  utilities  that  are  reasonably  good  approximations  of  the  expert’s  preference. 
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6.9  Summary 

This  chapter  provided  insight  as  to  how  knowledge  can  be  captured  in  a  probab- 
listic  manner  for  use  in  a  decision-analytic  system,  how  uncertainty  in  the  evidence  is 
represented  using  Jeffrey’s  Rule,  and  how  utility  values  are  assigned.  Also  discussed 
were  the  advantages  this  approach  offers.  Though  this  thesis  deals  with  manipulat¬ 
ing  probabilities,  it  has  not  concerned  itself  as  to  the  various  methods  with  which 
these  probilities  can  be  ascertained.  Several  papers  advocating  probabilistic  repre¬ 
sentation  of  knowledge  have  deeply  researched  this  area  with  algorithms  and  other 
heurestic  techniques  which  can  be  used  to  obtain  accurate  probabilities  from  expert 
knowledge.  Because  of  the  assumptions  that  this  method  is  best  applied  towards 
areas  which  already  lend  themselves  to  probabilistic  methods  due  to  large  historical 
data  bases,  known  frequencies,  etc.,  discussions  concerning  specific  techniques  to  ob¬ 
tain  probabilities  in  complex  domains  were  not  presented.  This  chapter  has  laid  the 
groundwork  for  the  next  chapter  in  which  the  method  will  be  applied  to  an  actual 
problem. 
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VII.  Problem  Application 


7.1  Introduction 

The  following  example  was  generated  as  a  test  case  to  demonstrate  the  utility 
of  the  decision-analytic  approach  to  rule-based  expert  system  development.  The 
example  concerns  satellite  anomaly  resolution,  and  serves  as  a  guideline  as  to  what 
information  is  required  in  the  application  of  the  methodology.  Given  the  fact  that 
on-orbit  anomalies  in  military  satellite  systems  or  subsystems  occur  quite  often,  and 
the  number,  frequency,  and  severity  of  the  anomalies  are  likely  to  grow  with  the 
inevitable  increases  in  spacecraft  complexity,  (17:207)  the  successful  application  of 
this  approach  to  this  type  of  problem  should  be  of  interest  to  the  Air  Force. 

7.2  Assumptions  in  Satellite  Anomaly  Resolution 

The  following  assumptions  apply  to  fault  diagnosis  in  satellite  operations. 

(a)  Time  between  failures  is  long  compared  to  diagnosis  time,  ruling  out  si¬ 
multaneous  failures  (which,  historically,  do  not  happen  in  this  enviroment). 

lb)  Sensors,  which  are  monitored  via  telemetry  can  either  be  the  cause  of  an 
anomaly  or  merely  indicate  one  has  occured.  Therefore,  information  from  adjacent 
values  can  be  used  to  determine  the  veracity  of  individual  sensor  readings. 

(c)  Past  trends  can  be  used  to  reliably  forecast  future  problems. 

7.3  Problem  Formulation 

The  situation  concerns  a  hypothetical,  yet  realistic,  occurence  in  GPS  satel¬ 
lite  operations  in  which  the  Loadshed  One  Timer  on  the  vehicle  is  unexpectedly 
timing  up.  Based  on  previous  experiences  and  historical  telemetry  data,  GPS  Elec¬ 
trical  Power  Subsystem  (EPS)  engineers  have  concluded  that  there  are  four  likely 
conditions  (hypotheses)  that  could  explain  the  sensor  reading. 
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These  conditions  are: 

1.  Eclipse  operations; 

2.  Attitude  problem  (loss-of-earth,  loss-of-yaw  control); 

3.  The  timer  is  faulty;  or 

4.  The  solar  arrays  are  not  tracking. 

Based  on  these  hypotheses,  the  appropriate  actions  are  either  going  to  be 

1.  Monitor  the  vehicle; 

2.  Carry  out  appropriate  attitude  contingency  procedure; 

3.  Disable  the  loadshed;  or 

4.  Perform  solar  array  contingency  procedures. 

In  order  to  ascertain  the  problem  and  narrow  the  hypothesis  set,  the  following  teleme¬ 
try  values  (evidence)  can  be  consulted: 

1.  Loadshed  2  is  also  timing  up; 

2.  SDV  (shunt  dissipation  voltage)  >  .3; 

3.  BCC  (Boost  Convert  Current)  <  .15; 

4.  ±  YSAC  (Solar  Array  Current)  summed  is  >  1.0; 

5.  Yawl  and  Yaw2  >  1.5  and  HZ  >  ±  .35; 

6.  Pitchl  and  Rolll  >  0.5  degrees;  and 

7.  YER  (Yaw  pointing  error)  >  2  degrees  or  YPOS  of  one  wing  is  >  ±  5  degrees. 

These  values  correspond  to  critical  points  the  GPS  Orbital  Operations  Hand¬ 
book  (OOH)  Sections  3.7  and  3.8  enumerates.  The  value  tolerance  levels  have  been 
determined  from  these  contingency  procedures  as  well  as  from  expert  input.  As 
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with  the  initial  assumptions  previously  mentioned,  the  hypotheses  form  an  exclu¬ 
sive  and  exhaustive  set  (i.e.,  the  problem  is  definitely  one  of  these  choices,  and  it 
is  not  a  combination  of  two  or  more  hypotheses).  Also,  only  one  action  will  be 
recommended.  In  real  world  satellite  operations  most  decisions  concerning  anomaly 
resolution  are  made  in  group  decisions.  So,  even  though  the  expert  system  would 
recommend  a  particular  course  of  action,  more  often  than  not  this  decision  would  be 
reviewed  before  any  action  is  initiated.  For  purposes  of  this  thesis,  the  goal  will  be 
the  recommendation  of  an  action  based  on  the  available  evidence. 

In  order  to  facilitate  testing  of  the  methodology  a  computer  program  coded 
in  PC  Scheme  was  developed  as  a  prototype  decision-analytic  reasoner.  The  pro¬ 
gram.  using  the  Quattro™-generated  file  as  the  knowledge  base,  allowed  for  varying 
degrees  of  evidence  belief  to  be  manipulated  in  a  manner  which  adhered  to  Bayes' 
theorem,  while  also  calculating  utility  values  in  a  decision  theoretic  manner. 

7.4  Knowledge  Base 

The  following  likelihoods  and  priors  were  determined  from  personal  interviews 
and  phone  conversations  with  Lt  Pam  Neal,  Master  Control  Station  (MCS)  EPS 
Engineer.  The  values  are  based  on  the  scenario  that  Loadshed  1  Timer  is  increasing 
(  Ed).  Suggested  actions  were  determined  from  the  GPS  OOH  as  well  as  inputs  from 
Lt  Neal. 

7.4-1  Priors.  The  prior  probabilities  of  the  hypotheses  were  determined  math¬ 
ematically.  Given  the  initial  scenario,  Lt  Neal  ranked  the  likelihood  of  possible  out¬ 
comes  in  relation  to  one  another.  It  was  determined  that  H2  (attitude  problem)  was 
the  least  likely  and  each  of  the  other  values  was  determined  in  relation  to  it.  It  was 
determined  that  the  likelihood  of  eclipse  operations  to  an  attitude  problem  was  500 
to  1;  a  faulty  timer  to  an  attitude  problem  was  1  to  1;  and  a  solar  array  tracking 
problem  to  attitude  problem  10  to  1.  Since  the  hypotheses  are  mutually  exclusive 


and  exhaustive,  the  probabilities  are  summed  and  the  unknowns  set  equal  to  1.  The 
solved  equations  yield  the  following  prior  probabilities: 

P[H1]  =  .9766 
P[H2j  =  .001953 
P[H3]  =  .001953 
P[H4]  =  .01953 

7.4.2  Likelihoods.  The  following  likelihood  data  was  determined  by  EPS  en¬ 
gineers  to  be  an  accurate  representation  of  the  data,  given  the  initial  scenario  of 
Loadshed  1  timing  up. 


Priors 

P [Ei | Hi] 

Eclipse 

P[Ei |H2] 

Attitude  problem 

P  CHI]  = 

.9766 

P [El | HI] 

=  .999 

PCE1IH2] 

=  .999 

P[H2]  = 

.001953 

P [E2 1  HI] 

*  .005 

P[E2|H2] 

=  .005 

P[H3]  * 

.001953 

PCE3IH1] 

*  .001 

P[E3|H2] 

*  .001 

P [H4]  = 

.01953 

P[E4|Hl] 

=  .9 

P[E4|H2] 

=  .99 

PCE5IH1] 

*  .81 

P[E5|H2] 

=  .92 

P [E6 I  Hi] 

=»  .0001 

PCE6IH2] 

*  .81 

P [E7 | Hi] 

=  .45 

P[E7|H2] 

■  .  58 

P[Ei |H3] 

Faulty  Timer  P[EilH4] 

Solar  Array  Prob 

PCE1IH3] 

»  .001 

P[E1 |H4] 

»  .999 

P[E2|H3] 

»  .989 

P[E2|H4] 

*  .005 

PCE3IH3] 

*  .999 

P[E3|H4] 

*  .001 

P[E4|H3] 

»  .9 

P  [E4 1 H4] 

-  .99 

PCE5IH3] 

*  .0001 

P[E5|H4] 

=>  .2 

PCE6IH3] 

*  .0001 

P[E6|H4] 

*  .02 

P[E7|H3] 

*  .001 

PCE7IH4] 

*  .99 

These  values  were  used  to  generate  the  joint  likelihood  matrix.  See  Appendix  E 
for  Quattro™  data  files. 
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7-4-3  Utilities  and  Actions  The  following  utility  values  were  determined  as 
accurate  representations  of  the  expert’s  belief  of  the  relative  utility  each  action  pro¬ 
vided  given  the  a  particular  hypothesis  (see  Figure  7.1).  Diagonal  values  of  100 
indicate  doing  the  correct  action  given  the  stated  hypothesis.  Lower  values  repre¬ 
sent  the  relative  utility  provided  by  an  action  given  an  inaccurate  assessment  of  the 
problem.  The  scaled  utility  table,  which  reflected  the  relative  importance  of  the 
hypotheses,  was  used  in  the  knowledge  base  (see  Section  6.8)  for  the  actual  problem 
application. 


HI 

H2 

H3 

H4 

Al 

100 

2 

50 

10 

A2 

0 

100 

0 

10 

A3 

40 

10 

100 

60 

A4 

10 

40 

10 

100 

Figure  7.1.  Unsealed  Utility  Table 


7.5  Validation  and  Testing  of  Methodology 

The  decision-analytic  methodology  was  tested  and  evaluated  using  the  GPS 
OOH,  Sections  3.7  -  3.8  which  concern  satellite  anomalies,  as  well  as  from  discussions 
with  the  MCS  GPS  EPS  satellite  engineer.  Comparing  the  model’s  output  against 
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the  expert's  recommended  actions  and  contingency  procedures  in  the  OOH  proved 
to  be  quite  successful. 

7.5. 1  Test  Methodology.  In  order  to  test  the  viability  of  accurately  represent¬ 
ing  GPS  system  knowledge,  several  computer  runs  were  done  with  a  PC  Scheme- 
coded  program  which  combines  information  in  a  decision-theoretic  manner.  This 
program  was  developed  as  a  prototype  inference  engine  which  reasoned  with  uncer¬ 
tainty  using  a  derivative  of  Bayes’  Rule  and  made  recommendations  according  to 
maximum  utility.  (See  Appendix  C  for  computer  code.)  The  utility  and  likelihood 
values  provided  by  the  expert  were  used  as  the  knowledge  base. 

Simulating  the  situation  of  seeing  the  timer  increase,  the  expert  was  asked; 
given  the  observance  of  certain  telemetry  points  (no  uncertainty  involved),  what 
would  he  surmise  the  problem  to  be  and  what  course  of  action  would  be  taken.  Be¬ 
cause  the  evidence  set  is  a  compilation  of  the  evidences  which  have  been  determined 
as  necessary  to  differentiate  between  the  different  hypotheses,  conflicting  subsets  of 
the  evidence  data  were  not  mixed.  This  would  avoid  giving  credance  to  multiple 
problems  occuring  simultaneously,  which  would  violate  one  of  the  assumptions  of 
this  thesis.  This  testing  was  done  in  order  to  confirm  that  the  assumptions  used 
in  combining  the  data  (i.e.,  independence)  would,  in  fact,  identify  highly  suspected 
problems.  It  was  also  used  to  validate  the  utility  values  and  likelihoods  in  order  to 
see  if  they  did  indeed  provide  answers  which  coincided  with  expert  opinion  and  GPS 
OOH  material. 

The  system  performed  quite  well,  correctly  identifying  suspected  problems  in 
all  cases  where  all  seven  of  the  telemetry  points  in  the  evidence  set  were  commented 
upon.  See  Appendix  F  on  computer  runs  to  show  actual  output  versus  expected 
output.  However,  some  interesting  facts  were  observed  which  showed  that  the  system 
had  apparently  captured  some  heuristic  expert  knowledge  that  would  not  show  up 
in  the  GPS  OOH.  In  particular,  when  Evidences  (1  and  6)  are  observed  as  true,  and 
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Evidences  (2  3  4  5  7)  are  observed  as  false,  the  correct  assessment  is  an  attitude 
problem  with  the  recommendation  to  start  attitude  contingency  procedures.  The 
DA  reasoner  coincided  with  this  assessment  outputting  a  normalized  utility  value  of 
about  63%  in  favor  of  an  attitude  problem.  However  what  is  interesting  is  the  fact 
that  if  the  belief  in  the  level  of  E-6  drops  below  .995,  the  suspected  hypothesis  is  an 
eclipse  (normal  operations).  The  expert  agreed  this  was  an  accurate  assessment  due 
to  the  fact  that  an  attitude  problem  is  so  rare,  you  would  almost  have  to  be  1 00 /T 
sure  of  the  telemetry  value  before  you  would  act  accordingly.  This  stands  in  direct 
conflict  with  the  GPS  OOH,  which,  written  under  the  guise  that  all  telemetry  values 
are  assumed  to  be  true,  recommends  that  appropriate  actions  be  taken  regardless  of 
an  operator's  subjective  feelings.  In  order  to  further  illustrate  the  manner  in  which 
the  model  had  mimmicked  the  expert’s  reasoning,  if  the  belief  in  E-6  drops  to  .9, 
the  normalized  belief  in  an  attitude  problem  drops  to  only  3%.  See  results  below  for 
corresponding  computer  output. 

This  shows  how  certain  one  must  be  of  the  loss  of  earth  values  before  appro¬ 
priate  action  would  be  taken. 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 


(((1  1)  (6  1) )  ( (2  1)<3  1X4  1X5  1X7  1))) 


2 . 2255475021483e-4 
0.00528862635  * 

6 . 6 109466042966e-4 
0.00227466014004297 


10.026347394230459 

10.62609997426044 

10.0782644343703208 

10.269288197138781 


Do  action  2 
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Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8) ) ( (3  .6))) 

(((1  1) (6  . 996) ) ( (2  1) (3  1) (4  1)(5  1)(7  1))) 


0.00427808700996717  10.293918175243603  I 
0.0052754674266  *  10.362441379983816  I 
0.00229851447977434  10.157915250459447  I 
0.00270329842840143  10.185725194313134  I 


Do  action  2 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  1) (6  . 9) ) ( (2  1) (3  1) (4  1)(5  1) (7  1))) 


0.101610861244023  10.630505693315243  I 
0.004959653265  *  10.0307751512207163  I 
0.0415965901440467  10.25811105808204  I 
0.0129906173490047  10.0806080973820009  I 


Do  action  1 


7 .5.2  Uncertainty  Testing.  Varying  the  belief  in  the  evidence  (uncertainty) 
proved  to  be  quite  interesting,  especially  when  conflicting  information  was  entered. 
The  expert  really  didn’t  know  how  to  approach  this  type  of  problem  because  of 
the  complexity  of  trying  to  determine  which  problem  to  act  on  or  how  to  logically 
combine  several  pieces  of  partial  information.  This  is  precisely  the  type  of  situation  in 
which  expert  systems  can  be  used  to  augment  operator  knowledge  by  recommending 
actions  which  will  maximize  the  expected  utility. 

Given  the  expert’s  inability  to  quantify  her  actions  given  certain  hypothetical 
situations,  several  runs  were  done  showing  exactly  how  sensitive  the  system  was  to 
varying  degrees  of  belief.  Though  several  cases  were  done  in  which  the  belief  in 
the  evidence  ranged  from  .1  to  1,  the  expert  noted  that  given  the  reliability  of  the 
satellite  equipment,  it  is  unlikely  that  belief  would  ever  drop  below  .8.  Also,  due  to 
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the  fact  that  a  .5  belief  in  a  value  is  also  a  .5  disbelief  in  it  (indifference),  it  would 
not  make  sense  logically  to  use  values  below  this  mark.  Appendix  G  is  a  listing  of 
a  few  computer  runs  which  show  how  slight  variations  in  the  levels  of  uncertainty 
can  cause  a  small  change  in  the  output  utility  values  or  a  large  change.  Because 
of  the  rational  nature  in  which  the  system  operates,  the  actions  recommended  will 
maximize  the  expected  utility. 

7 .5.3  Sensitivity.  Sensitivity  testing  was  done  in  order  to  see  exactly  how  ro¬ 
bust  the  model  was  given  the  variability  of  the  prior  and  utility  information.  Three 
scenarios  were  involved  in  the  testing:  the  first  concerned  a  suspected  attitude  prob¬ 
lem,  the  second  concerned  a  faulty  timer,  and  the  third  involved  a  solar  array  prob¬ 
lem.  The  eclipse  operations  hypothesis  was  not  tested  because  under  the  scenario 
that  the  data  was  developed,  this  would  indicate  normal  operations. 

7.5.4  Prior  Sensitivity.  In  the  prior  test,  three  sets  of  prior  values  were  tested 
with  the  scaled  utility  data  in  order  to  determine  how  sensitive  the  output  was 
to  change.  First,  the  expert  provided  data  was  used  as  a  gold  standard  in  order 
to  measure  any  noticeable  change  given  the  variability  of  the  data.  The  first  test 
involved  modifying  the  ratios  provided  the  expert  which  were  used  to  calculate  the 
initial  priors.  Instead  of  predicting  that  the  eclipse  operations  are  500  times  more 
likely  than  the  attitude  problem,  the  number  was  halved  to  250.  The  ratio  of  the 
solar  array  problem  was  doubled  from  10  to  20  times  more  prevalent  while  the  ratio 
between  the  attitude  and  faulty  timer  problems  stayed  the  same.  Using  these  ratios, 
the  new  prior  values  are: 

HI  =  .9191,  H2  =  .00367,  H3  =  .00367,  and  H4  =  .07353.  See  Section  2.4.1  for 
formula  used  to  solve  equation.  Test  Two  involved  changing  the  prior  values  to  (.75 
05  .05  .15)  while  the  Test  Three  set  all  values  equal  to  .25. 
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These  tests  concern  varying  the  priors  with  Scenario  1,  the  attitude  problem. 
This  first  test  is  the  gold  standard  with  the  expert’s  priors  and  utility  values. 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (C(l  .8)) ((3  .6))) 

(((1  1) (5  1) (6  1) ) ( (2  1) (3  1) (4  1)  (7  1))) 

0.00164951870500002  10.0177405542122364  I 

0.060703116705  (0.652861304023074  I 

0  00626584711000004  10.0673891117473198  I 

0.02436162877  10.26200903001737  I 


Do  action  2 

This  test  is  with  prior  values  of  (.9191  .00367  .00367  .07353)  Though  the  values  have 
been  modified  accordingly,  the  effect  is  minimal  with  all  values  maintaining  roughly 
the  same  percentages. 


0.00270210990500004  10.0155071975041286  I 
0.114078195705  10.654685846951901  I 
0.0116564311100001  10.0668953468848772  I 
0.04581202127  10.262911608659094  I 


Do  action  2 

This  test  is  done  using  prior  values  of  (.75  .05  .05  .15)  and  scaled  utilities.  A  more 
dramatic  change  of  the  priors  still  results  in  minimal  change  of  the  values.  The 
attitude  problem  still  has  about  66%  of  the  utility  yet  its  prior  is  25  times  higher 
than  it  originally  was. 


0.0314412750000006  10.0133050227978924  I 
1.554029775  10.657619692107851  I 
0.155711250000001  10.0658924210655677  | 
0.6219309  10.263182864028689  I 


Do  action  2 
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This  test  was  done  with  all  the  priors  equal  to  (.25  .25  .25  .25)  Even  with  equal 
priors,  no  change  is  really  noticeable  in  this  scenario.  I  attribute  this  to  both  the 
likelihood  values  and  the  high  utility  values  associated  with  the  attitude  problem. 


0.155560125000003 

7.770049625 

0.777341950000006 

3.1085073 


10.0131702717674423  | 
10.657839952287012  | 
10.0658125257853416  | 
10.263177250160204  I 


Do  action  2 


These  prior  tests  were  done  on  Scenario  2,  the  faulty  timer.  This  is  with  the  expert 
provided  priors  and  scaled  utilities.  The  timer  has  62.2%  of  the  utility  while  the 
eclipse  has  31%,  the  attitude  problem  zero  percent  and  the  solar  array  6%. 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 


(((1  1)  (2  1)  (3  1) )  ( (4  1)  (5  1)  (6  1)  (7  1))) 


1 .978629044814e-4 
1 .005795e-8 
3 . 87584931087e-4 
3 . 9102776028e-5 


10.316803337336471  1 
1 1 . 61040399923114e-5  I 
10.620572107700108  I 
10.0626084509234294  I 


Do  action  3 


This  test  was  done  with  priors  of  (.9191  .00367  .00367  .07353)  With  the  mod¬ 
ified  priors,  the  values  stay  essentially  the  same. 


3 . 67041348026e-4 
2 . 611918e-8 
7 . 26464722810001e-4 
7.30743719200001e-5 


10.314623078595486 
1 2 . 23890220166899e-5 
10.622716129424492 
10.062638402958005 


I 

I 


Do  action  3 
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This  test  was  done  with  priors  of  (.75  .05  .05  .15).  This  dramatic  change  in 
the  priors  still  results  in  roughly  the  same  proportional  utilities. 


0.00493894759000001  10.31261844663589  I 
1 . 889e-7  1 1 . 19567222557872e-5  | 
0.00987175835000001  10.624848452976168  | 
9 . 87749300000001e-4  10.0625211436656868  I 


Do  action  3 

This  test  was  done  with  priors  of  (.25  .25  .25  .25)  Even  with  equal  priors,  the  timer 
problem  is  still  identified  with  62%  of  the  utility.  Just  like  the  attitude  problem  it 
appears  as  if  the  likelihood  values  combined  with  the  utilities  are  determining  the 
result. 


0.02467636995  10.312501314443063  I 
8 . 465e-7  1 1 . 07200679521362e-5  I 
0.0493508957500001  10.624979274588059  i 
0.0049359395  10.0625086909009255  | 


Do  action  3 

This  is  Scenario  3,  the  solar  array  problem,  with  the  experts  original  priors  and  the 
scaled  utilities.  The  solar  array  problem  has  52%  of  the  utility  with  the  eclipse  at 
10%,  the  attitude  problem  at  5%  and  the  timer  at  33%. 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8)) ((3  .6))) 

(((1  1) (4  1) (7  1))((2  1) (3  1) (5  1) (6  1))) 


14.9160660502193  10.102468532262818  I 
7.62023493  10.0523485405700168  t 
47.7074220910387  10.327734504744941  I 
75.3235620340039  10.517448422422224  I 


Do  action  4 
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This  was  done  with  priors  of  (.9191  0.00367  0.00367  0.07353)  The  slight  change 
in  the  priors  caused  the  eclipse  to  become  the  recommended  action  with  a  value  of 
55%.  The  solar  array  goes  to  44%  while  both  the  attitude  and  the  timer  drop  to 
zero. 


0.0702238355 
3. 180789e-5 
3 . 6333e-14 
0.05610544884 


Do  action  1 


10.555739383545053  I 
1 2 . 5 1722 1831 392 11 e-4  I 
I  2 . 87533127157978el3  I 
10.44400889427152  I 


This  is  with  the  priors  of  (.75  .05  .05  .15).  With  these  priors,  once  again  the  solar 
array  is  the  recommended  action  with  66%  of  the  utility.  The  eclipse  drops  from 
55%  to  33%. 


0.05730375  10.332791203735817  | 
4 . 3335e-4  10.0025166776718612  I 
4 . 95000000000001e-13  I  2 . 87470969786846e-12  I 
0.1144542  10.664692118589447  | 


Do  action  4 

This  is  with  equal  priors  of  (.25  .25  .25  .25).  With  equal  priors  the  selection  of  the 
solar  array  problem  is  pronounced  with  almost  90%  of  the  utility.  Eclipse  goes  from 
33%  to  9%  while  attitude  increases  from  zero  to  1. 


10.900896120729963  I 
10.0102193137600505  I 
1 1.16731517508365e-ll  I 
10.89969107415528  I 


Do  action  4 


0.01910125 

0.00216675 

2.4756-12 

0.190757 


7. 5.5  Utility  Sensitivity.  In  order  to  test  the  effects  of  utility  values  on  the 
recommendations  of  certain  actions  given  the  evidence,  three  tests  were  done  in  each 
scenario.  Test  One  involved  the  expert  provided  data  in  which  the  scaled  utility 
values  were  used.  This  served  as  the  standard  with  which  other  values  could  be 


7-13 


compared.  Test  Two  involved  using  the  unsealed  utility  values  which  were  provided 
by  the  expert  to  see  if  the  scaling  actually  had  any  noticeable  effect  on  the  output. 
Lastly  in  Test  Three,  the  posterior  probabilities  would  be  examined  in  order  to  see 
what  if  any  different  action  would  be  pursued  given  the  knowledge  of  the  likelihood 
and  prior  data  without  utility  data.  As  in  the  prior  test,  these  tests  were  run  on 
scenarios  that  would  indicate  an  attitude  problem,  a  faulty  timer  problem  and  a 
solar  array  problem. 

These  tests  concern  varying  the  priors  with  Scenario  1,  the  attitude  problem. 
This  first  test  is  the  gold  standard  with  the  expert’s  priors  and  utility  values.  In  this 
case  the  attitude  action  has  a  65.2%  value  associated  with  it. 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  1)  (5  1) (6  1) ) ( (2  1) (3  1) (4  1) (7  1))) 

0.00164951870500002  10.0177405542122364  | 

0.060703116705  10.652861304023074  I 

0.00626584711000004  10.0673891117473198  I 

0.02436162877  10.26200903001737  ! 


Do  action  2 

This  is  with  the  unsealed  utility  values  provided  by  the  expert.  In  this  case  the  belief 
in  the  attitude  problem  drops  from  65%  to  about  39%  while  the  eclipse  action  (1) 
has  climbed  from  1.7%  to  28%.  The  solar  array  problem  dropped  from  26%  to  18% 
and  the  timer  increased  from  6%  to  14%. 


4.43874582100001e-4  10.282451031895904  I 
6.0706993410-4  10.386297247542986  I 
2.33827324600002e-4  10.14879150954368  I 
2.867380216-4  10.18246021101743  I 


Do  action  2 
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This  is  with  only  the  posterior  values  used.  No  utility  information.  Though 
the  attitude  problem  is  still  recommended  with  an  increased  value  of  58%,  the  strong 
prior  associated  with  the  eclipse  hypothesis  (.9766)  causes  this  value  to  go  up  to  41% 
while  the  timer  and  solar  array  problems  went  to  zero. 


4 . 316572e-6 
6 . 069924e-6 
2. 1483e-20 
7 . 75341e-9 


Do  action  2 


10.415284628041265  I 
1 0 . 583969  *39309422  i 
I  2 . 0668 1 590489 178e- 15  I 
1 7 . 45932 6493 10941 e-4  I 


This  is  Scenario  2,  the  faulty  timer.  Values  used  are  those  provided  by  the  expert. 
In  ths  case  the  timer  value  is  62%,  the  eclipse  31%,  the  attitude  problem  essentially 
zero,  and  the  solar  array  about  6%. 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  1 )  (2  1) (3  1 ) ) ( (4  1) (5  1) (6  1) (7  1))) 


1 ,978629044814e-4 
1 ,005795e-8 
3 . 87584931087e-4 
3 . 9102776028e-5 


10.316803337336471  I 
1 1.61040399923114e-5  1 
10.620572107700108  I 
10.0626084509234294  | 


Do  action  3 


This  is  with  the  unsealed  utility  values.  In  this  case  the  timer  goes  down  to  55% 
while  the  eclipse  increases  from  31%  to  38%.  The  attitude  problem  goes  even  lower 
while  the  solar  array  stays  about  the  same. 


1 ,4735984803614e-5 
1 ,388583e-10 
2. 13l571637567e-5 
2 . 43818669628e-6 

Do  action  3 


10.382852028280071 
I  3 . 607643636 10666e-6 
10.553798429994754 
10.0633459340815384 
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This  is  with  only  the  posterior  values.  The  timer  is  identified  with  a  strong 
1 9%  value.  I  attribute  this  to  these  3  evidences  being  very  descriptive  of  a  faulty 
timer  anomaly.  The  eclipse  drops  to  21%  while  both  the  attitude  and  solar  array 
problems  are  zero. 


5.09785200000001e-8  10.209144449071664  | 
6 . 23007e- 13  I  2 . 555948l8725201e-6  | 
1 . 927611e-7  i 0 . 790821586463239  | 
7 . 65576e- 12  1 3 . 14085169091783e-5  | 


Do  action  3 

This  is  Scenario  3,  the  solar  array  not  tracking.  Values  used  are  those  provided  by 
the  expert.  In  this  case  the  the  distribution  is  rather  spread  out.  The  solar  array 
problem  hoids  just  over  half  the  utility  with  52%.  The  timer  is  next  with  33%  while 
the  eclipse  and  attitude  problems  are  10%  and  5%,  respectively. 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8) ) ( (3  .6))) 

(((1  1 ) (4  1) (7  1) ) ( (2  1) (3  1) (5  1) (6  1))) 


14.9160660502193  10.102468532262818  I 
7.62023493  10.0523485405700168  I 
47.7074220910387  10.327734504744941  I 
75.3235620340039  10.517448422422224  I 


Do  action  4 

This  is  with  the  unsealed  utility  values.  Without  the  scaling,  the  eclipse  scenario  is 
now  the  choice  with  55%  while  the  solar  array  problem  goes  to  16%.  This  is  quite 
a  dramatic  turnaround.  I  attribute  this  to  strength  of  the  prior  for  the  eclipse  and 
also  because  Evidences  1  and  4  also  lend  belief  to  an  eclipse.  The  attitude  problem 
drops  to  1%  while  the  time-  problem  goes  to  28%. 


7.61076552i70296  10.548425223791249  I 
0.1507120335  i 0 . 0108601796316775  I 
3.87897039691193  10.279515273718491  I 
2.23704198004019  10.161199322858582  I 


Do  action  1 
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This  is  with  only  the  posterior  values.  The  dominant  prior  of  the  eclipse  causes 
it  to  be  selected  with  83%  of  the  total  utility.  The  solar  array  problem  is  still  about 
the  same  (16%)  while  both  the  timer  and  the  attitude  problem  drop  to  zero. 


10.833375784690598  I 
1 1.89048578826992e-4  I 
1 2 . 15943340300822e-13  I 
10.166435166730359  I 


Do  action  1 


0.074617123 
1 .6926651e-5 
1 ,93347e-14 
0.01490193684 


7.5.6  Stength  of  Likelihoods  I  believe  that  the  very  minor  changes  observed 
in  some  of  the  output  given  the  variability  of  the  prior  and  utility  data  indicates 
the  strength  that  is  contained  in  the  likelihood  data.  I  also  attribute  this  lack  of 
variation  to  the  dissimilarity  of  the  problems  and  the  discriminating  evidences.  If 
you  examine  the  cases  below  you  can  see  that  in  all  three  cases,  given  the  observed 
values,  the  likelihood  values  are  in  the  90%  range  of  the  suspected  problem  with  the 
attitude  and  faulty  timer  values  being  99%.  Consequently  the  utilty  values  would 
have  to  be  ridiculously  large  or  the  prior  values  so  insignificant  in  order  to  effectively 
overcome  the  strength  of  the  likelihood.  Had  there  been  two  problems  in  which 
the  evidences  were  the  same  yet  the  likelihoods  were  slightly  different,  the  effect 
of  varying  the  priors  or  utilities  would  have  a  much  more  pronounced  effect.  This 
confirms  my  belief  that  the  strength  of  the  implication  in  the  rules  which  operate  in 
this  type  of  system  are  contained  in  the  likelihood  data. 

This  is  the  likelihood  value  of  the  attitude  problem. 


10.00141993570454029  I 
*  10.998452527083985  I 
I3.53377664025863e-15  I 
11.275372114711536-4  I 


Do  action  2 


1 . 105e-6 
7 . 77e-4 
2 . 75e- 18 
9 . 925e-8 
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This  is  the  likelihood  values  for  the  faulty  timer  problem. 


1 . 305e-8  I  5 . 28592012847095e-4  I 

7 . 975e-ll  1 3 . 23028452295446e-6  | 

2 . 4675e-5  *  |0 . 999464208199392  I 

9 . 8e-ll  1 3 . 9695032382387 le-6  | 

Do  action  3 

This  is  the  likelihood  values  for  the  solar  array  problem. 

0.01910125  10.0900896120729963  I 

0.00216675  10.0102193137600505  I 

2 . 475e-12  I  1 . 16731517508365e-ll  I 

0.1907s/’  *  10.89969107415528  I 

Do  action  4 

The  robustness  of  the  system  can  be  attributed  to  the  combination  of  the 
uKelihood,  priors,  and  utility  values.  Given  the  fact  that  the  expert  provided  values 
are  fairly  accurate,  varying  the  prior  or  utility  information  still  will  usually  result  in 
the  expected  outcome.  Further  testing  would  have  to  be  done  in  order  to  see  if  this 
robustness  would  hold  given  varying  degrees  of  belief  in  the  evidence  or  changes  in 
the  likelihood  data. 

The  expert  noted  that  determining  the  likelihoods  was  the  easiest  of  all  the 
subjective  evaluations.  Consequently,  I  feel  fairly  safe  in  attributing  the  overall 
robust  nature  of  .he  system  to  the  strength  of  the  likelihoods.  Judging  from  this 
example,  it  appears  that  if  the  likelihoods  give  an  accurate  representation  of  the 
data,  the  priors  and  the  utility  values  can  compensate  for  the  inaccuracy  of  one 
another  allowing  the  system  to  still  yield  favorable  results.  Further  testing  in  different 
situations  would  have  to  be  done  before  this  hypothesis  could  really  be  substantiated. 


7.6  Functional  Relationship  of  Evidences 

Because  the  DA  model  represents  and  manipulates  the  system  knowledge  in 
functional  in  manner,  a  few  evidences  were  isolated  and  graphically  depicted  in  or¬ 
der  to  examine  this  relationship.  This  type  of  information  can  be  used  in  sensitivity 
analysis  to  determine  which  values  have  the  most  effect  on  the  action.  Looking  at 
the  two  telemetry  points  El  and  E3,  (Figure  7.2)  shows  how  these  values  relate  func¬ 
tionally  in  the  decision  space.  (Figure  7.3)  shows  the  complex  relationship  between 
E 123  and  E7.  Points  falling  along  the  lines  separating  the  different  action  spaces  in¬ 
dicate  values  where  the  user  would  be  indifferent  between  the  two  actions.  Although 
Morlan  (13)  recommends  determining  these  functions  and  writing  rules  which  repre¬ 
sent  their  information,  this  research  did  not  because  of  the  complexity  involved  with 
accurately  defining  such  functions  and  because  of  the  limited  flexibility  the  system 
would  have  due  to  the  complexity  of  altering  any  of  the  values.  Furthermore,  though 
this  might  be  possible  in  two-space  given  the  ability  to  graph  the  function,  as  the 
number  of  evidences  increased  beyond  three,  the  ability  to  graphically  represent  the 
decision  space  would  disappear. 


.1  .3  .5  .7  .9 

E3 


Figure  7.2.  Graph  of  El  vs  E3 
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E7 

Figure  7.3.  Graph  of  E123  vs  E7 


7.7  Summary 

Comparing  the  output  of  the  computer  program  given  the  data  provided  by 
the  expert  showed  that  it  is  possible  to  accurately  model  an  expert’s  knowledge 
in  a  probabilisitic  manner.  The  output  provided  by  the  decision-analytic  reasoner 
not  only  agreed  with  checklist  procedures  in  the  GPS  OOH,  but  also  demonstrated 
the  ability  to  reason  in  a  manner  which  matched  the  expert’s  given  the  validity  of 
the  evidence.  The  mathematical  nature  in  which  the  model  represented  the  rules 
provided  the  ability  to  graphically  depict  the  functional  relationship  of  the  evidences 
showing  which  were  the  most  sensitive,  as  well  as  which  were  the  most  robust  given 
certain  values.  Though  expected  predictions  were  difficult  to  make  given  uncertainty 
of  the  evidence,  I  operated  under  the  premise  was  that  if  the  system  provided  rational 
output  which  agreed  with  the  expert’s  assessment  under  certainty,  it  would  also 
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provide  rational  decisions  under  uncertainty  or  with  conflicting  information.  The 
robustness  inherent  in  a  decision-analytic  model  is  no  doubt  a  function  of  the  prior, 
likelihood,  a  utility  data.  In  this  case  due  to  the  dissimilarity  of  the  hypotheses, 
the  likelihood  data  played  the  dominant  factor  in  determining  actions.  However, 
given  the  domain  and  the  scenario  in  which  this  model  was  developed  this  was  to  be 
expected. 

Though  the  decision-analytic  reasoner  developed  to  test  the  application  of  the 
methodology  is  functionally  correct,  it  nonetheless  is  a  rough  prototype  used  merely 
to  show  the  capability  of  using  decision  theory  as  a  manner  of  capturing  system 
knowledge  while  also  reasoning  coherently  and  consistently  with  uncertainty. 


VIII.  Conclusions  and  Recommendations 


8.1  Introduction 

The  purpose  of  this  thesis  was  to  demonstrate  the  applicability  of  using  a  deci¬ 
sion  theory  approach  towards  reasoning  in  expert  systems.  Particular  emphasis  was 
placed  on  the  management  of  uncertainty  and  how  it  could  be  consistently  factored 
into  a  decision.  Using  a  Quattrorvi-generated  file  as  a  knowledge  base  and  a  Scheme 
pc-coded  program  as  a  decision-analytic  reasoner,  this  capability  was  demonstrated 
using  an  anomalous  condition  onboard  the  GPS  satellite  as  the  test  scenario.  The 
model  was  able  to  effectively  capture  GPS  system  knowledge  probabilistically  in  a 
manner  that  matched  expectations  of  the  expert  as  well  as  coincided  with  the  GPS 
00H.  Though  the  application  is  complex,  it  is  nonetheless  a  valid  method  for  build¬ 
ing  an  expert  system,  given  the  application  domain  satisfies  the  assumptions  and 
constraints  under  which  the  model  is  based. 

8.2  Key  Concepts 

Probability  was  used  as  the  representation  for  uncertainty  because  of  its  solid 
theoretic  basis,  with  a  derivative  of  Bayes'  Rule  being  used  in  the  actual  problem 
application.  This  allowed  coherent  manipulation  of  the  evidence  belief  values  in  a 
manner  that  adhered  to  the  laws  of  probability,  yet  allowed  for  subjective  interpre¬ 
tation.  Utility  assessments  allowed  the  user  the  ability  to  rank  the  importance  of 
certain  actions  in  terms  of  his  own  value  scale.  Combined  together,  the  two  areas 
provided  a  rational  framework  in  which  sound  decisions  could  be  made. 

8.3  Lessons  Learned 

Though  the  application  methodology  is  valid,  it  is  nonetheless  a  complex  pro¬ 
cess  due  to  the  consistent  value  assessments  of  probability  and  utility  that  must  be 
provided  by  the  expert. 
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In  order  to  develop  a  database  which  could  realistically  manipulate  several 
different  classes  of  problems  would  be  very  time  consuming,  not  to  mention  extremely 
hard  to  control.  Though  the  detail  needed  by  any  large-scale,  rule-based  expert 
system  would  be  extensive,  a  system  founded  on  this  type  of  reasoning  scheme  would 
require  a  more  rigorous  validation  process  due  to  the  functional  nature  of  the  rules. 

Though  the  assumptions  of  conditional  independence  of  evidences  and  mutual 
exclusiveness  of  hypotheses  help  with  computations,  they  nonetheless  limit  potential 
areas  of  application.  The  restrictive  nature  of  the  single  recommended  actions  might 
also  be  unrealistic  in  some  real  world  applications.  Nevertheless,  as  stated  at  the 
onset  of  the  thesis,  this  type  of  knowledge  base  reasoning  is  more  suited  towards 
problems  that  lend  themselves  to  probabilistic  representation  and  involve  uncertainty 
reasoning. 

8.4  Sensitivity  of  Belief  Assessments 

In  this  scenario,  decision  theory  proved  to  be  a  viable  expert  system  reason¬ 
ing  methodology,  however  the  successful  application  of  this  inference  method  was 
nonetheless  highly  dependent  on  the  expert’s  assessment  of  the  probabilities  and 
utilities.  Had  these  values  been  different,  the  results  could  have  been  less  successful. 

Though  I  agree  that  probability  is  the  only  logical  representation  for  uncer¬ 
tainty  in  expert  systems,  there  is  a  serious  question  concerning  its  application  and 
assessment  in  real  world  decision  making.  This  is  not  referring  to  its  manipulation 
(this  thesis  showed  how  that  can  be  done),  but  rather  how  and  to  what  situations  it 
can  be  applied. 

For  example,  if  I  develop  an  expert  system’s  data  base  with  my  subjective 
opinions  and  beliefs,  the  system  is  biased  towards  my  assessments  which  may  or 
may  not  correlate  with  another  expert’s  or  a  particular  user’s.  This  bias  is  woven 
into  the  likelihood  and  utility  data  and  directly  effects  the  system  output.  Conse¬ 
quently,  problems  could  arise  in  situations  in  which  my  belief  in  an  evidence  value 


is  .9  and  yet  a  user's  belief  is  .7.  Given  the  disparity,  a  different  course  of  action 
could  be  pursued  which,  given  the  scenario  under  which  the  system  was  developed, 
was  inadvertantly  recommended.  Therefore,  a  reasoning  system  of  this  sort  must 
also  provide  restrictions  as  to  under  what  situations  a  user  can  question  the  verac¬ 
ity  of  the  information.  Otherwise,  though  the  system  recommended  action  will  be 
rational  given  the  data  it  is  working  with,  it  may  or  may  not  coincide  with  what  the 
expert  would  have  done  in  the  same  situation.  With  a  system  of  this  sort,  added 
responsibility  comes  with  the  added  versatility. 

5.5  Summary 

This  thesis  demonstrated  that  a  decision-theoretic  approach  towards  rule- based 
expert  systems  offers  both  strength  in  theory  and  application.  With  satellite  anomaly 
resolution  as  a  model,  a  prototype  decision-analytic  reasoner  was  developed  which 
manipulated  uncertainty  in  a  sound  probabilistic  manner  and  combined  it  with  utility 
to  rationally  recommend  courses  of  action.  Though  it  is  by  no  means  close  to  being  a 
functional  expert  system,  it  nonetheless  demonstrates  the  underlying  concepts  that 
make  decision  analysis  both  a  logical  and  rational  choice  for  further  development  in 
expert  system  reasoning  schemes. 

5.6  Recommendations 

Though  decision  theory  is  unquestionably  a  valid  reasoning  methodology  for 
expert  system  development,  this  thesis  raised  a  few  questions  which  need  to  be 
answered  in  order  to  further  determine  the  validity  and  useability  of  this  approach 
in  larger  more  complex  problem  domains.  The  following  are  recommendations  which 
can  help  a:hieve  this  goal: 

1.  Expand  the  DA  reasoner  to  include  nested  hypotheses.  This  would  allow  the 
system  to  still  work  with  small  evidence  sets  yet  would  permit  much  larger  problems 
to  be  handled. 


2.  Explore  new  and  more  efficient  ways  of  determining  probabilities.  This 
could  involve  probabilistic  knowledge  acquisition  tools  such  as  belief  networks  or 
influence  diagrams.  This  would  further  insure  that  knowledge  obtained  from  the 
expert  would  be  more  representative  of  his  actual  beliefs. 

3.  Determine  a  quick  method  in  which  a  user  can  determine  exactly  which 
values  have  the  most  effect  on  the  output  (i.e.,  priors,  likelihoods,  or  utilities).  This 
can  help  in  fine  tuning  the  system  in  order  to  assist  with  development  and  validation. 

4.  Combine  a  decision  theoretic  rule-base  with  a  model-based  expert  system. 
A  hybrid  system  like  this  would  offer  much  more  than  either  a  rule-based  or  model- 
based  system  could  alone. 
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Appendix  A.  GPS  Subsystems 


The  following  paragraphs  give  a  summary  of  each  of  the  GPS  satellite  subsystems. 


A.l  Structure  Subsystem 

The  structure  subsystem  consists  of  the  following: 

1.  Primary  and  secondary  structure 

2.  Mechanisms 

a.  Solar  array  deployment 

b.  Forward  TT&C  mast  deployment 

c.  Passive  nutation  damper  release. 

The  primary  and  secondary  structure  are  completely  passive  and  neither  receive  commands 
nor  produce  telemetry  outputs.  The  solar  arrays  are  folded  along  the  SV  sides  during  launch  and 
initial  orbital  operations  and  are  deployed  when  configuring  the  vehicle  for  three-axis  stablization. 
If  the  vehicle  is  launched  from  the  shuttle,  the  forward  TT&tC  mast  deployment  is  provided  by  the 
PAM-D  II  after  separation  from  the  orbiter.  The  passive  nutation  damper  is  released  by  ground 
control  command  shortly  after  first  acquistion. 

A. 2  Thermal  Control  Subsystem 

The  Thermal  Control  Subsystem  (TCS)  provides  space  vehicle  and  equipment  temperature 
•ontrol  within  accepted  limits  during  all  mission  phases.  Passive  thermal  control  is  accomplished 
by  thermal  coatings,  insulation  blankets,  thermal  shields  and  heat  transfer  doublers.  Active  ther¬ 
mal  control  is  provided  by  heaters,  thermal  switches,  and  frequency  standard  active  baseplate 
temperature  control  units  (ARTCU).  Heat  rejection  is  provided  by  semi-active  louvers  and  direct 
radiators  which  are  maintained  parallel  to  the  sun  line  by  the  AVCS  and  by  passive  battery  radi¬ 
ators  Minimum  temperatures  for  the  EPS  batteries,  OIS,  LBS,  and  the  wetted  RCS  component 
are  maintained  by  heaters  and  thermostat  controls. 


A. 3  Electrical  Power  Subsystem 

ne  Electrical  Power  Subsystem  (EPS)  provides  for  the  generation,  storage,  control,  and 
distribution  of  electrical  power  to  all  space  vehicle  subsystems.  The  EPS  is  capable  of  supplying 
all  required  power  for  steady-state  and  transient  loads. 

The  solar  arrays  provide  primary  electrical  power  during  all  phases  of  mission  operations 
following  sun  acquisition.  Three  nickei-cadmium  batteries  are  also  charged  by  the  solar  arrays  to 
provide  power  during  eclipses,  and  during  the  launch  and  orbit  injection  phases  when  insufficient 
power  is  available  from  the  stowed  arrays.  In  addition  to  the  solar  arrays  and  batteries,  the  EPS 
includes  power  conditioning  equipment  for  the  bus  power  conditioning  and  battery  charging,  a 
load  control  unit  for  controlling  and  distributing  power  to  the  loads,  a  solar  array  drive  and  power 
transfer  assembly  for  positioning  the  solar  arrays  and  transferring  the  power  to  the  main  bus,  and 
a  DC/DC  converter  for  providing  secondary  DC  voltages. 
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A. 4  Attitude  and  Velocity  Control  Subsystem 

The  Attitude  and  Velocity  Control  subsystem  (AVCS),  in  conjuction  with  the  other  SV 
subsystems,  provides  for  space  vehicle  (SV)  attitude  control  and  velocity  corrections  and  control 
subsequent  to  separation  from  the  PAM-D  II  to  satisfy  the  operational,  thermal,  electrical  power, 
and  payload  and  subsystem  pointing  requirements.  The  AVCS  also  provides  active  nutation  damp¬ 
ing  prior  to  SV  separation  from  the  PAM-D  II  and  passive  nutation  damping  during  subsequent 
periods  of  spin-stabilized  operations. 

The  SV  has  two  stablization  modes — an  inertially  stablized  spin  mode  and  a  three-axis 
stablized  mode  with  the  SV  +Z  axis  aligned  along  the  earth  nadir.  The  spin-stablizied  mode 
is  capable  of  SV/PAM-D  II  or  ELV  launch  separation  stabliziation,  long-term  on-orbit  storage, 
attitude  sensing  for  attitude  determination,  precession  maneuvers,  and  large  (5-lbf)  thruster  drift 
orbit  corrections.  The  three-axis  stabilized  mode  provides  all  on-station  attitude  control,  including 
the  capability  to  acquire  the  earth  and  sun,  provide  payload  and  space  vehicle  pointing,  perform 
unrestricted  small  (0.1-lbf)  orbit  corrections,  and  stationkeeping. 

Attitude  control  is  accomplished  either  autonomously  or  from  the  ground  with  override  of 
autonomous  functions.  Velocity  maneuvers  are  ground  controlled  and  support  drift  orbit  injections 
correction,  final  orbit  insertion,  orbit  phasing,  and  orbit  period  adjustment. 

The  three- axis  stabilized  space  vehicle  maintains  earth  nadir  and  solar  array  pointing  through 
a  combination  of  yaw  steering  and  soiar  array  orientation.  The  single  degree-of-freedom  solar  array 
drives  about  the  space  vehicle  (SV)  Y-axis  (pitch  axis)  and  the  space  vehicle  yaw  steers  about  the 
Z-axis  (local  vertical)  The  space  vehicle  yaws  in  orbit  to  maintain  the  sun  in  the  space  X-Y  plane 
by  using  the  yaw  sun  sensor  located  on  the  solar  arrays.  Nadir  sensing  is  accomplished  with  a 
static  earth  sensor  The  solar  array  is  maintained  normal  to  the  sun  line  with  the  pitch  sun  sensor 
mounted  on  the  solar  arrays. 


.4.5  Orbit  Insertion  Subsystem 

The  Orbit  Insertion  subsystem  (OIS)  consists  of  a  single  solid  rocket  motor  which  provides 
rhe  required  impulse  to  boost  the  space  vehicle  from  the  transfer  orbit  into  an  initial  drift  orbit. 
OIS  ignition  is  controlled  by  ground  comrrand  based  on  mission  planning  activities  supported  by 
software  resident  at  the  Air  Force  Mission  Control  Center  (MCC). 


.4.6'  Reaction  Control  Subsystem 

The  Reaction  Control  subsystem  (RCS)  prov’des  tht  necessary  impulse  to  perform  SV  trans¬ 
lation  and  spin  vector  precession  maneuvers  during  the  SV/PAM-D  II  transfer  orbit  operations, 
SV  drift  orbit  operations,  and  SV  on-orbit  operations.  The  RCS  is  a  blowdown  system  utilizing 
monopropellant  hydrazine  pressurized  by  gaseous  nitrogen.  The  subsystem  consists  of  a  propel- 
lant/pressurant  storage  (PPS)  assembly,  a  propellant  distribution  and  control  (PDC)  assembly,  and 
the  rocket  engine  assembly  module  (REAM). 

The  PPS  components  consist  of  two  propellant  tanks,  each  equipped  with  a  positive  expulsion 
diaphram,  two  propellant  fill  and  drain  valves,  two  pressurant  fill  valves,  and  two  temperature 
transducers. 

The  PDC  components  consist  of  two  pressure  transducers,  two  filters,  and  two  latching 
isolation  valves. 

The  REAM  components  consist  of  twenty  0.44-N  (0.1-lbf)  thrusters  and  two  22.2-N  (5- 
lbf)  thrusters,  each  equipped  with  a  temperature  sensor,  thruster  valve  heaters,  and  catalyst  bed 
heaters  The  5-pound  thrusters,  in  conjuction  with  the  AVCS,  are  used  to  perform  SV  translation 


A- 2 


and  spin  vector  precession  maneuvers  during  spin-stabilized  operations.  The  0.1-pound  thrusters 
are  used  to  perform  translation  (stationkeeping  and  rephasing)  maneuvers  and  provide  backup  SV 
attitude  control  during  three-axis  stabilized  operations. 


.4.7  Telemetry,  Tracking  and  Commanding  Subsystem 

The  Telemetry,  Tracking  and  Commanding  (TTizC)  subsystem  performs  telemetry,  track¬ 
ing  and  commanding  functions,  as  required,  to  support  space  vehicle  and  launch  vehicle  (PAM-D 
1 1 /Or  biter )  operations;  and  is  compatible  with  the  Air  Force  Satellite  Control  Facility  (AFSCF), 
the  Space  Ground  Link  System  (SGLS)  and  the  Operational  Control  System.  The  IT&rC  sub¬ 
system  supports  the  integration,  checkout  and  launch  of  the  SV  in  the  PaM-D  11/Orbiter  launch 
vehicle  and  supports  the  upload  of  processor  data,  RFDU,  NDU  and  DCEA,  via  the  TTCC  re¬ 
ceiver/demodulators  and  signal  conditioner  from  the  OCS.  The  TT&C  subsystem  receives  com¬ 
mands  via  the  uplink  from  the  OCS  RTSs  and  processes  this  data  for  control  of  SV  subsystems. 

The  TT&'C  subsystem  also  collects,  processes  and  transmits  SV  telemetry  data  including 
Nav  igation  Data  Unit  (NDU)  memory  dump  data  and  is  capable  of  supporting  non-coherent  pseudo¬ 
random  noise  (PRN)  turnaround  ranging. 


A.  3  Navigation  Payload 

The  Navigation  Payload  (NPD)  consists  of  four  Frequency  Standard  Assemblies  (FSA),  the 
Frequency  Synthesizer  and  Distribution  Unit  (FSDU),  and  the  Navigation  Data  Unit  (NDU).  The 
Navigation  Payload,  in  its  primary  capacity,  generates  pseudo-random  noise  codes  and  modulo-two 
adds  navigation  data  to  them  prior  to  their  output  to  the  L-band  subsystem  where  they  modulate 
the  L-band  carrier  frequencies 

A. 9  L-Dand  Subsystem 

The  L-band  subsystem  (LBS)  consists  of  the  devices  required  to  generate,  amplify,  filter, 
combine,  and  transn.it  signals  at  LI,  L2,  and  L3  carrier  frequencies.  Interface  devices  from  accep¬ 
tance  of  uplink  commands  from,  and  exchange  of  data  with  the  TT&C  subsystem  are  included  for 
control,  update,  and  function  monitoring  of  the  L-band  subsystem. 


A.  10  Nuclear  Detonation  Detection  System 

The  Nuclear  Detonation  (NUDET)  Detection  System  (NDS)  consists  of  the  Integrated  Trans¬ 
fer  Subsystem  (ITS)  and  the  Global  Burst  Detector  (GBD). 

A.  10.1  Integrated  Transfer  Subsystem  (ITS).  The  Integrated  Transfer  Subsystem 
(ITS)  provides  the  NUDET  Detection  System  (NDS)  with  a  satellite-to-satellite  UHF  communi¬ 
cation  link.  The  ITS  provides  a  data  relay  to  other  space  vehicles  by  means  of  a  time-division, 
multiple-access  (TDMA)  UHF  crosslink  data.  The  TDMA  mechanization  provides  a  UHF  crosslink 
lata  15  second  time  slot  for  each  ITS  equipped  space  vehicle  and  accomodates  up  to  24  satellites 

A.  10.2  Global  Burst  Detector  (GBD).  The  Global  Burst  Detector  (GBD)  consists 
of  the  processor  and  those  sensors  used  in  the  detection  of  nuclear  detonations  The  following 
■  omponents  constitute  the  GBD: 

1  Burst  Detector  Processor  (BDP) 
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2  Burst  Detector  Optical  Sensor  (BDY) 

3  Burst  Detector  X-Ray  Sensor  (BDX)  or 

1  Burst  Detector  Dosimeter  (BDD)-  SV''s  18,  24,  28,  33,  and  39. 

A.  10.3  GPS  Cargo  Element  Should  the  satellite  be  launched  from  the  shuttle,  the 
SV  itself  is  a  subsystem  of  the  launch  vehicle.  The  SV,  PAM-D  II,  and  cradle  assembly  create  a 
cargo  element  in  the  space  shuttle’s  cargo  bay. 

This  concludes  a  brief  summary  of  the  GPS  subsystems. 
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Appendix  B.  Program  Outline 


The  scheme  program  TEST3.S  was  written  in  order  to  demonstrate  the  appli¬ 
cability  of  uncertainty  management  and  utility  theory  in  expert  system  development. 
The  program  is  approximately  600  lines  long  and  serves  as  a  prototype  inference  en¬ 
gine  of  a  decision  analytic  reasoner.  The  program  accesses  a  separately  loaded  file 
which  serves  as  the  knowledge  base  of  the  particular  problem  domain.  This  knowl¬ 
edge  base  contains  the  priors,  the  joint  likelihood  table,  and  the  utility  action-matrix. 
The  priors  are  the  user  provided  values  which  are  written  in  a  scheme  construct  called 
"priors".  This  is  how  the  prior  values  .6.  .2,  .1,  and  .1  would  be  represented. 


(define  (priors) 

' (.6  .2  .1  .1)  ) 

The  joint  likelihood  table  is  a  Quattro™  generated  file  which  has  been  altered 
in  order  to  fit  into  scheme  syntax.  This  data  file  is  labeled  “likelihoods'’.  Below  is  a 
hypothetical  example. 


(define  (likelihoods) 

'(  ((()  (1  2  3))  (.1  .1  .1  .1)) 

(((1)  (2  3))  (.2  .6  .3  .4)) 


(((1  2  3)  ())  (.02  .024 


042  .024))  )) 


The  action-matrix  defines  the  utility  values  which  the  expert  has  determined 
for  each  action.  The  list  is  constructed  with  the  different  actions  representing  the 
different  rows  and  the  possible  hypotheses  of  the  problem  domain  representing  the 
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different  columns.  Below  is  how  the  action  matrix  is  represented  in  the  knowledge 
base. 


HI 

H2 

H3 

H4 

A1 

100 

10 

20 

15 

A  2 

5 

100 

6 

30 

A3 

70 

80 

100 

10 

A  4 

10 

20 

30 

100 

(define  (action-matrix) 
’(  (100  10  20  15) 

(5  100  6  30) 

(70  80  100  10) 
(10  20  30  100)  )) 


After  loading  TEST3.S  and  the  appropriate  file  containing  the  priors,  likeli¬ 
hoods,  and  action-matrix  values,  the  system  is  ready  for  use.  Below  is  the  step-by- 
step  process  for  using  the  program. 

Step  1.  type:  pcs 

This  loads  scheme  program  into  the  computers  resident  memory.  (This  assumes 
the  scheme  program  is  resident  on  the  default  drive.) 

Step  2.  type:  (load  ua:test3.fsr ) 

This  loads  the  fastload  version  of  the  TEST3.S  from  the  floppy  drive  into  the 
scheme  buffer. 


Step  3.  type:  (load  “a:knowledgebase”) 

This  loads  the  user  created  knowledge  base.  This  file  name  is  whatever  the 
user  has  named  it. 


Step  4.  type:  (start) 
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This  starts  the  program,  asking  the  user  to  indicate  what  evidence  he  has 
observed.  First,  enter  the  evidence  values  that  were  observed  as  true  and  the  belief 
in  that  evidence.  Then  enter  the  evidences  that  were  observed  as  false  and  their 
corresponding  belief  value. 


T  F 

(((1  .3) (2  . 1) ) ( (4  . 1) (7  .8))) 

In  the  above  example.  Evidence  1  and  2  were  observed  as  true  and  4  and  7 
as  false.  With  a  belief  in  Evidence  1  of  .3  and  a  belief  in  Evidence  2  of  .1.  The 
associated  beliefs  in  Evidences  4  and  7  are  .1  and  .8,  respectively.  The  ‘true*  set 
consists  of  the  subsets  (1  .3)  and  (2  .1),  while  the  false  subsets  are  (4  .1)  and  (7  .8). 
Plcc.~e  note  that  there  must  be  a  space  in  between  the  evidence  and  the  belief  value. 
The  belief  values  can  range  from  0  to  1. 

After  the  values  are  entered,  hit  the  return  key.  The  program  will  now  take  the 
entered  values  and,  utilizing  a  combination  of  Bayes’  Theorem  and  decision  theory, 
calculate  the  appropriate  utility  values  for  each  action.  The  action  yielding  the 
highest  utility  will  be  recommended  by  the  system  as  the  rational  course  of  action. 
A  normalized  list  will  also  output  the  percentage  of  utility  associated  with  each 
action.  Action  values  are  vertically  output  in  ascending  order  starting  from  the  top. 


A1 
A2 
A3 
A4 

Do  action  3 


10 

30 

40 

20 
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In  order  to  validate  the  accuracy  of  the  output  data,  several  data  runs  were 
accomplished  testing  the  approximately  35  subroutines  that  are  accessed  during  the 
programs  operation. 

The  program's  calculations  can  be  broken  up  into  6  major  steps.  Following  is 
a  short  description  of  each  step.  Step  1  breaks  the  entered  evidences  into  all  possible 
subset  combinations. 


(i.e.  ,(12  3)  ->  (0(1  2  3)) 

((1)  (2  3)) 
((2)  (1  3)) 
((3)  (1  2)) 
(O  2)  (30 
((1  3)  (2)) 
((2  3)  (1)) 
((1  2  3)0)  ) 


Step  2  takes  each  component  list  and  retrieves  the  associated  likelihood  values 
from  the  joint  likelihood  matrix. 


(i.e.  ,(12  3)  ->  (0(1  2  3)) 

(.1 

.1 

.  1 

.1) 

((1)  (2  3)) 

(.2 

.6 

.3 

.4) 

((2)  (1  3)) 

(.5 

.1 

.2 

.2) 

((3)  (1  2)) 

(.2 

.4 

.7 

.3) 

((1  2)  (3)) 

(.1 

.06 

.06 

.08) 

((1  3)  (2)) 

(.04 

.24 

.21 

.12) 

((2  3)  (1)) 

(.1 

.04 

.14 

.06) 

((1  2  3)0) 

(.02 

.024 

.042 

.024) 

Step  3  takes  each  component  list  and  calculates  the  corresponding  belief  vector. 
This  vector  is  determined  by  the  placement  of  the  evidences  in  the  sublist  and  their 
original  placement  as  entered  by  the  user.  For  example,  if  the  values  are  in  the  same 
category  (i.e.,  true  or  false)  as  entered  by  the  user,  the  entered  belief  values  are  used. 


If  the  values  are  in  opposite  list,  the  entered  belief  value  substracted  from  1  is  used. 
For  example  if  the  entered  list  is 


T  F 

<origmai  user  entered  list>  (((l  . 3 )  ( 2  - 6) )  ( (3  .8))) 


<Example 

component  sublist> 


T  F  resultant  vector 
((1  2)  (3))  ( .3) C -6) C .8)  =  ( .3) ( .6) ( .8) 
( (2  3)  (I))  C .6) Cl- .8) Cl- -3)  =  ( .6) ( .2) C -7) 


These  belief  vectors  are  multiplied  by  the  likelihood  values  in  ordcr  to  give 
their  respective  weights. 


<Example 

HI 

H2 

H3 

H4 

component  sublist> 

((1 

2) 

(3)) 

(.1 

.OS 

.06 

.08) 

*  (.144) 

((2 

3) 

(l)) 

(.1 

.04 

.14 

.06) 

*  (.084) 

The  resultant  likelihood  values  are: 


HI  H2  H3  H4 

((1  2)  (3))  (.0144  .00864  .00864  .01152) 
((2  3)  (1))  (.0084  .00336  .01176  .00504) 


Step  4  adds  all  of  the  likelihood  values  together  to  get  the  total  scaled  likelihood 
associated  with  each  hypothesis.  Though  I  am  only  showing  two  likelihoods,  this 
value  would  include  all  subset  lists  of  the  entered  evidences. 


HI  H2  H3  H4 

((1  2)  (3))  (.0144  .00864  .00864  .01152) 
((2  3)  (1))  (.0084  .00336  .01176  .00504) 


.0228  .0348  .0204  .01656 
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Step  5  multiplies  each  of  the  hypothesis  likelihood  values  by  its  respective  prior 
in  order  to  determine  the  posterior  value  (assume  priors  are  (.6  .2  .1  .1)). 


HI  H2  H3  H4 

(1  2)  (3)  ( . 0228) ( . 6)  ( .0348) (  .2)  (.0204)(.l)  (,01656)(.l) 

(1  2)  (3)  (.01368)  (.00696)  (.00204)  (.001656) 


Step  6  takes  the  resultant  posteriors  and  does  a  dot  product  with  each  row  of 
the  action-matrix  in  order  to  determine  the  utility  values. 


01368  .00696  .00204  .001656]  *  I  100 

10 

20 

15  i 

=  1.50324 

1  5 

100 

6 

30  1 

=  .82632 

1  70 

80 

100 

10  I 

=  1.73496 

1  10 

20 

30 

ioo! 

=  .5028 

The  largest  value  is  selected  and  the  associated  action  is  recommended.  In  this 
case  it  is  Action  3.  The  values  are  also  normalized  and  the  relative  percentage  of 
each  utility  value  is  output. 

Presently  as  it  stands,  the  hypotheses  and  actions  are  limited  to  4  values  while 
there  is  no  limit  on  the  number  of  evidences  which  can  be  used  in  the  knowledge  base. 
This  thesis  used  7  evidences  which  meant  during  each  run,  128  different  sublists  werp 
manipulated.  As  the  number  of  evidences  increases,  the  time  determining  the  output 
will  vary  due  to  the  factor  of  two  that  is  added  for  each  evidence. 
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Appendix  C.  Scheme  Code 


Decision-Analytic  Code.  The  language  is  SCHEME 
Written  by  Grady  Elliott  for  Thesis  work.  1-8  Oct  1989 


PRIORS 

These  are  the  prior  values  of  the  hypothesis  set.  They 
should  be  put  into  the  particular  problem  data  set. 


; (define  (priors) 

;  '(.6  .2  .1  .1)) 


LIKELIHOODS 

This  shows  the  form  in  which  the  likelihood  values  from 
the  p{e}lh  need  to  entered  in.  The  likelihoods  show  be 
put  into  the  particular  problem  data  set.  (These  have  been 
commented  out.) 


(define  (likelihoods) 


((() 

(1 

2 

3)) 

( 

.  1 

.  1 

.  1 

■  D) 

(((1) 

(2 

3)) 

( 

.2 

.6 

.3 

.4)) 

(((2) 

(1 

3)) 

( 

.5 

.  1 

.2 

.2)) 

(((3) 

(1 

2)) 

( 

.2 

.4 

.7 

•  3)) 

(((1 

2) 

(3)) 

( 

.  1 

.06 

.06 

.08)) 

(((1 

3) 

(2)) 

( 

.04 

.24 

.21 

.12)) 

(((2 

3) 

(1>) 

( 

.  1 

.04 

.14 

.06)) 

(((1 

2 

3) 

()) 

(. 

,02 

.024 

.042 

.024))  )) 

(define  (values) 

’(((1  . 9) (2  . 8) ) ( (3  .7)(4  .8)))  ) 


ACTIOI-MATRIX 

This  is  the  action  utility  diagram.  Each  row  is  for  a 
particular  action,  1-4.  Use  particular  matrix  for  each 
problem  data  set.  (Example  below  has  been  commented  out) 


(define  (action-matrix) 
’((100  10  20  15) 

(5  100  8  30) 

(70  80  100  10) 

(10  20  30  100))) 


ACTIO! 

This  procedure  recommends  1  of  4  actions  depending  upon 
the  utility  values  of  the  results. 


(define  (action  1st) 

(cond  ((null?  1st)  '()) 

((equal?  (car  1st)  (car  (reverse  (sort  1st)))) 
(display  "Do  action  1")) 

((equal7  (cadr  1st)  (car  (reverse  (sort  1st)))) 
(display  "Do  action  2")) 

((equal?  (caddr  1st)  (car  (reverse  (sort  1st)))) 
(display  "Do  action  3")) 

(else 

(display  "Do  action  4")))) 


ADD 

This  procedure  adds  values  in  a  vector  to  obtain  the  total 
resultant  values.  Used  when  all  evidence  is  not  observed. 


(define  (add  lstl  lst2) 

(cond  ((null?  Ist2)  lstl) 

(else 

(append(list  (+  (car  lstl)  (car  lst2)>) 
(add  (cdr  lstl)  (cdr  lst2)))))) 


START  (BGH) 

This  procedure  starts  the  whole  program. 

Ask  user  to  input  the  evidences  he  observed  and  the 
belief  in  that  evidence. 


(define  (start) 

(bgn) 

(res«t)) 

(define  (bgn) 

(window-clear  ’console) 

(window-delete  ‘percent-window*) 

(display  "Enter  evidences  observed  true  and  false  and 
Belief  m  that  evidence  reading,  i.e.  (((1  8))((3  .6)))  ") 

(newline) 

(newline) 

(let*  ((inputl  (read)) 

(input2  inputl)) 

(myprop  input2) 
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(total 
(posterior 
( Digaad 
(big-build 

(d-total(do-all(flat(extract  inputl)))  (likelihoods)) 
(extract  inputl)))  (priors))  (action-matrix))  )) 


BIG-ADD 

This  procedure  is  used  to  compute  total  values  of  the 
likelihood  vectors  after  the  BIG-BUILD  routine.  It  takes 
one  list  of  many  vectors  and  outputs  a  single  vector 
with  the  sum  of  all  the  sublist  vectors. 

[263]  (bigadd  '((1  2  3)  (1  2  3)  (1  2  3))) 

=>(369) 


(define  (bigadd  1st) 

(cond  ((null7  1st)  '()) 

(else 

(add  (car  1st)  (bigadd  (cdr  1st)))))) 


BIG-BUILD 

This  procedure  is  used  on  the  output  of  D-TOTAL.  Its 
purpose  is  to  output  a  list  of  vectors  ehich  have 
been  adjusted  to  the  value  determined  by  the  DETERMIIE  and 
MULT  procedures.  It  accesses  the  SCALAR  routine  and  BIG2 


(define  (bl6-build  1st  goodlst) 

(cond  ((null?  1st)  '()) 

(else 
( append 

(list  (scalar  (cadar  1st)  (big2  (caar  1st)  goodlst))) 
(big-build  (cdr  1st)  goodlst;)))) 

(define  (big2  lstl  lst2) 

(cond  ((null?  lstl)  '()) 

(else 

(mult  (determine  lstl  lst2))))) 


3UILD 

This  procedure  builds  the  double  list  needed  by  joining  the 
incoming  list,  with  a  list  of  the  reverse  of  the  members  in 
in  each  li.,'.  (  C  ( i )  (2  3))  ((3)  (5  6))  +  ((2  3)  ( 1) )  (5  6) 
(3))  ) 


(define  (build  1st)  ;builds  double  the  list  reversing  the  order 
(cond  ((null7  1st)  ’()) 

( else 

(append  1st  (flip  1st))))) 


CHECK 

(old)  This  procedure  was  used  when  I  initially  didn’t  take 
uncertainty  of  evidence  into  consideration.  It  adds  the 
vector  values  if  the  evidence  meets  criteria. 


(define  (check  lstl  lst2) 

(cond  ((null7  lst2)  ’()) 

((subset7  lstl  (caaar  lst2))  (add  (cadar  lst2)  (check  lstl  (edr  lst2)))) 
( else 

(check  lstl  (edr  lst2))))) 


CLIP 

This  procedure  builds  sets  of  head  to  tail  throughout  the  list 
(1  2  3  4  6  6)  =>  (1  8)  (2  5)  (3  4) 


(define  (clip  1st) 

(cond  ((null7  1st)  ’()) 

(else 
( append 
(list 

(append  (list  (car  1st))  (list  (car  (reverse  1st))))) 
(clip  (remove  (edr  1st))))))) 


D-CHECK 

This  procedure  used  to  check  to  if  I  can  build  proper 
subset  list  (i.e.  (evidences  +  (summed  likelihoods)) 
car  of  the  do-all  list  and  likelihood  list.  It  accesses 
the  MEWCHECK  procedure. 


(define  (d-check  lstl  lst2) 

(append  (list  lstl)  (list  (newcheck  lstl  lst2))  )) 
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D-TOTAL 

This  procedure  produces  the  hugh  list  of  the  different 
combinations  with  the  sum  total  of  the  likelihood  values 
generated  in  the  HEVCHECK  procedure. 


(define  (d-total  lstl  lst2) 

(cond  ((null7  lstl)  '()) 

(else 

(append 

(list  (d-check  (car  lstl)  lst2))  (d-total  (cdr  lstl) 
lst2))))) 


CELETE 

This  procedure  deletes  a  member  from  a  list. 


(define  (delete  x  1st) 

(cond  ((null7  1st)  '()) 

((equal7  x  (car  l3t))  (delete  x  (cdr  1st))) 
(else 

(cons  (car  1st)  (delete  x  (cdr  1st))  )))) 


DETERMINE 

This  procedure  checks  to  see  if  the  placement  of  the 
values  of  the  list  in  question,  matches  that  of  the  list 
created  by  the  user  with  the  read  statement.  It  determines 
the  values  of  the  evidence  that  are  going  to  be  used  to 
multiply  by  the  posterior  values.  These  values  are  retrieved 
from  the  property  list  which  has  assigned  the  values 
depending  upon  the  user  read  input.  If  placement  doesn't 
match,  then  one  minus  the  value  is  given.  A  list  of  the 
evidence  vector  is  output.  DETERMIIE2  is  consulted. 


(define  (determine  1st  goodlst) 

(append  (determine2  (car  1st)  (car  goodlst)) 

(determine2  (cadr  1st)  (cadr  goodlst)))) 

(define  (determine2  1st  goodlst) 

(cond  ((null?  1st)  ’()) 

((null?  (car  1st))  ’()) 

((member  (car  1st)  goodlst) 

(cons  (getprop  'grady  (car  1st)) 

(determme2  (cdr  1st)  goodlst))) 

(else 

(cons  (-  1  (getprop  'grady  (car  1st))) 

(determine2  (cdr  1st)  goodlst)))  )) 
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DO-ALL  combines  the  output  o f  the  BUILD,  CLIP,  and  SUBSETS 
routines.  It  gives  a  list  broken  down  into  all  of  its  subsets 


(define  (do-all  1st) 

(cond  ((null?  1st)  ’()) 

(else 

(build  (clip  (subsets  1st)))))) 


DOT 

This  procedure  figures  the  dotproduct  of  two  one-row  vectors 


(define  (dot  lstl  ls-c2) 

(cond  ((null?  lstl)  0) 

((null?  Ist2)  0) 

(else 

(+  (*  (car  lstl)  (car  lst2))  (dot  (cdr  lstl)  (cdr  lst2)))))) 


DOTP 

This  procedure  figures  the  dot  product  of  a  1  one-row  vector 
and  a  multirow  matrix.  Produces  a  matrix  list. 


(define  (dotp  lstl  lst2) 

(cond  ((null?  lstl)  ’()) 

((null?  Ist2)  '()) 

(else 

(append  (list  (dot  lstl  (car  lst2)))  (dotp  lstl  (cdr  lst2)))))) 

; ;  EXTRACT 

;  This  procedure  is  used  to  generate  the  list  of  evidences 
;;  without  the  subjective  belief  assessments. 

;;  (  ((1  .9)  (2  .8))  ((3  .4)  (4  .7)))  =>  ((1  2)  (3  4)) 

; ;  It  accesses  the  EXT1UCT2  procedure . 

(define  (extract  1st) 

(append  (list  (extract2  (car  1st))) 

(list  (extracts  (cadr  1st)))  )) 

(define  (extract2  1st) 

(cond  ((null?  1st)  '()) 

(else 

(cons  (caar  1st) 

(extract2  (cdr  1st))  )))) 


FLAT 

This  procedure  links  the  FLATTEH  and  FLATTEH2  procedures 
together . 


(deline  (Hat  1st) 

(flatten2  (flatten  1st))) 


FLATTEI 

This  procedure  flattens  out  a  list  of  many  sublist  into 
one  major  list.  ((1)(2)(3))  =>  (1  2  3) 


(define  (flatten  1st) 

(cond  ((null?  1st)  '()) 

((atom?  (car  1st)) 

(append  (list  (car  1st))  (flatten  (cdr  1st)))) 

(else 

(append  (flatten  (car  1st))  (flatten  (cdr  1st)))  ))) 


FLATTEH2 

This  procedure  is  used  to  alleviate  problems  which  arise 
when  the  flattened  list  has  nil  as  a  member,  (i.e. 

'(1  2  ()).  Heeds  to  be  done  before  the  DO-ALL  procedure. 
This  is  for  cases  when  either  no  evidence  was  true  or  no 
evidence  was  false. 


(define  (flatten2  1st) 

(cond  ((null?  1st)  '()) 

((member  '()  1st)  (delete  ’()  1st)) 
(else 
1st))) 


FLIP 

This  procedure  reverses  the  members  inside  of  a  list. 
((1  2)  (3))  =  >  ((3)  (1  2)) 


(define  (flip  set) 

(cond  ((null?  set)  ’()) 

(else 
( append 

(list  (reverse  (car  set)))  (flip  (cdr  set)))))) 
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LARGEST 

This  procedure  finds  the  largest  number  in  a  list 
hopefully  I  will  be  able  to  use  this  in  the  ACTIO! 
routine . 


(define  (largest  1st) 

(cond  ((null?  1st)  ’()) 

((null?  (cdr  1st))  (car  1st)) 

((>  (car  1st)  (cadr  1st))  (largest  (cons(car  1st)  (cddr 
1st)))) 

(else 

(largest  (cdr  1st))))) 


MULT 

This  procedure  takes  the  members  of  a  vector  and  multiplies 
them  by  one  another  to  get  the  SCALAR  value. 

[3]  (mult  '(2  3  4))  =>  24 


(define  (mult  1st) 

(cond  ((null?  1st)  1) 

(else 

(*  (car  1st)  (mult  (cdr  1st)))))) 


MYPRQP 

This  builds  a  property  list  which  keeps  track  of  the 
belief  of  each  of  the  evidences.  It  accesses  MYPR0P2 
[39]  (myprop  ■(  ((1  .9)  (2  .8))  ((3  .7)  (4  .2))  )  ) 
returns  =>  (0.9  0.8  (4  0.2)) 

[42]  (getprop  'grady  3) 

0.7 

[43]  (getprop  ’grady  1) 

0.9 

[44]  (getprop  'grady  2) 

0.8 

[45]  (getprop  'grady  4) 

0.2 


(define  (myprop  1st) 

(append  (myprop2  (car  1st))  (myprop2  (cadr  1st)))) 

(define  (myprop2  1st) 

(cond  ((null?  1st)  ’()) 

(else 

(cons  (putprop  'grady  (cadar  1st)  (caar  1st)) 
(myprop2  (cdr  1st)))))) 
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NEWCHECK 

This  procedure  adds  vector  values  if  the  evidence  maats 
the  correct  position  criteria.  LST2  is  LIKELIHOODS  values 


(define  (nescheck  lstl  lst2) 

(cond  ((null?  Ist2)  ’()) 

((and  (subset?  (car  lstl)  (caaar  lst2))  (subset?  (cadr  lstl)  (cadaar  lst2))) 
(add  (cadar  lst2)  (nescheck  lstl  (cdr  lst2)))) 

(else 

(newcheck  lstl  (cdr  lst2))))) 


PERCE8T 

This  procedure  is  used  to  tell  the  percentage  each 
member  makes  up  of  the  list.  It  accesses  the  PERCEIT2 
and  SUMLIST  procedures. 

i.e.  (percent  ’(2  2  2  2))  =>  (.25  .25  .25  .25) 


(define  (percent  1st) 

(let  ((total  (sumlist  1st))) 

(percent2  1st  total))) 

(define  (percent2  1st  lst2) 

(cond  ((null?  1st)  •()) 

(else 

(cons  (  /  (car  1st)  lst2) 

(percent2  (cdr  1st)  lst2))))) 


POSTERIOR 

This  procedure  takes  the  output  from  BIGADD  and  figures 
out  the  posterior  values  of  each  of  the  hypotheses.  This 
i3  the  vector  that  will  be  used  in  the  calculation  of 
the  utility  values  through  the  action  matrix. 


(define  (posterior  lstl  lst2) 

(cond  ((null?  lstl)  ’()) 

((null?  Ist2)  '()) 

(else 

(append  (list  (*  (car  lstl)  (car  lst2)))  (posterior  (cdr 
lstl)  (cdr  lst2)))))) 
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PREFIX 

This  procedure  is  used  to  help  fathom  all  the  subsets  of 
a  particular  list 


(define  (prefix  elt  1st)  ;a  first  element  to  help  expand  set 
(cond  ((null?  1st)  ’()) 

(else 

(cons  (cons  elt  (car  1st)) 

(prefix  elt  (cdr  1st))  )))) 


REMOVE 

This  procedure  takes  the  tail  off  of  a  list.  This  routine  is 
accessed  by  the  CLIP  to  build  the  needed  lists. 


(define  (remove  1st) 

(cond  ((null?  1st)  '()) 

(else 

(reverse  (cdr  (reverse  1st)))  ))) 


SCALAR 

This  procedure  is  used  to  take  the  multiplication  number 
given  by  the  DETERMINE  and  MULT  and  multiply  it  across  the 
the  vector  given  the  REVCHECK  procedure. 

[105]  (scalar  ’(123456)  100) 

(100  200  300  400  500  600) 

Then  this  vector  will  be  added  with  all  of  the  SUBSETS  vectors 
which  will  finally  be  used  in  the  POSTERIOR  routine. 


(define  (scalar  1st  x) 

(cond  ((null?  1st)  ’()) 

(else 

(cons  (*  (car  1st)  x)  (scalar  (cdr  1st)  x))))) 


SORT 

This  procedure  sorts  a  list  from  lowest  to  highest  values. 
It  accesses  the  IISERT  procedure  which  places  the  sorted 
values . 


(define  (sort  1st) 

(cond  ((null?  1st)  '()) 

(else 

(insert  (car  1st)  (sort  (cdr  1st)))  ))) 
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(define  (insert  x  sorted-list) 

(cond  ((null?  sorted-list)  (list  x)) 

((<  x  (car  sorted-list))  (cons  x  sorted-list)) 

(else 

(cons  (car  sorted-list)  (insert  x  (cdr  sorted-list)))))) 


SUBSET? 

This  procedure  determines  if  sets  are  subsets  of  each  other 


(define  (subset?  lstl  lst2) 

(or  (null?  lstl) 

(and  (member  (car  lstl)  lst2) 

(subset?  (cdr  lstl)  lst2)))) 


SUBSETS 

This  procedure  finds  all  the  subsets  of  a  list. 
It  access  the  prefix  routine. 

(subsets  ’ (1  2  3)) 

(()  (3)  (2)  (2  3)  (1)  (1  3)  (1  2)  (1  2  3)) 


(define  (subsets  set)  ;finds  all  subsets  of  a  set  of  values 
(cond  ((null?  set)  (list  nil)) 

(else 

(append  (subsets  (cdr  set)) 

(prefix  (car  set) 

(subsets  (cdr  set))  ))))) 


SUMLIST 

This  procedure  adds  the  members  of  the  list  to  give  the 
total  value.  It  is  used  in  the  PERCEIT  procedures, 
(sumlist  ’(123  4))  =>  10 


(define  (sumlist  1st) 

(cond  ((null?  1st)  0) 

(else 

(+  (car  1st)  (sumlist  (cdr  1st)))))) 
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TOTAL 

This  procedure  takes  the  dot  product  of  hypothesis  set  and 
ACTIOH-MATRIX  values  and  outputs  action  as  determined  in  the 
ACTIOI  routine. 


(deiine  (total  lstl  lst2) 

(cond  ((null?  lstl)  ’()) 

((null?  Ist2)  '()) 

(else 

(vertprint  (dotp  lstl  lst2)) 

(uindow-set-size !  *percent-Bindow*  4  22) 
(srindoa-set-position!  *percent-Bindow*  5  22) 
(uindoB-clear  *percent-sindos*) 

(set-fluid!  output-port  *percent-Bindow*) 
(vertprint  (percent  (dotp  lstl  lst2))) 
(set-fluid1  output-port  'console) 

(neuline)  (newline) 

(action  (dotp  lstl  lst2))))) 

(define  *percent -window*  (make-window  #f  #t)) 


VERTPRIIT 

This  procedure  vertically  prints  out  a  list. 


(define  (vertprint  1st) 

(cond  ((null?  1st)  *the-non-printing-object*) 
(else 

(print  (car  1st)) 

(vertprint  (cdr  1st))))) 


; ;  The  following  procedures  were  used  as  test  procedures  to 
; ;  check  and  validate  the  code. 


(define  (small-action) 

’((1  2  3  4) 

(2341) 

(2222) 

(1  1  1  1))) 

(define  (test)  ;  test  routine  to  determine  if  total  routine  uorked 

(print  "Enter  4  values  of  the  hypothesis  set")  ;  with  user  input  read 
(total  (read)  (action-matrix)))  ;  statement. 


GROUPLIST 

This  is  a  practica  hypothesis/likelihood  table  to  check 
numbers . 


(define  (grouplist) 

'(  (((1)  (2  3))  (123  4)) 
(((2)  (1  2))  (123  1)) 
(((1  2)  (3))  (1  1  1  D)  )) 


SMALl-ACTIOM 

This  is  a  practice  action  table  to  check  numbers. 


(define  (small-action) 
’((1  234) 

(2341) 

(2222) 

(1  1  1  1))) 
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Appendix  D.  GPS  Knowledge  Base 


The  following  listing  is  the  knowledge  base  accessed  by  the  DA  reasoner  (Test3.s 
The  values  used  for  the  priors,  likelihoods,  and  utilities  are  those  provided  by  the 
user,  written  in  scheme  syntax.  The  Quattrorw-generated  file  was  used  as  the  source 
for  the  likelihood  values. 


(define  (priors) 

•<  9766  0.001953  0.001953  0.01953)  ) 

(define  (action-matrix) 


(100 

200 

1000 

500) 

(0 

10000 

0 

500) 

(40 

1000 

2000 

3000) 

(10 

4000 

200 

5000)  )) 

(defiflo  ( likelihoods) 
'( 


((0(1  2 

3 

4 

5 

6 

7)) 

(1 . 04e-5 

6. 35e-8 

l.le-6 

7 . 79e-8) ) 

(((1) 

(2  3 

4 

5 

6 

7)) 

(0.010376 

6 . 34*-S 

l.le-9 

7 . 79e-5) ) 

(((2) 

(1 

3 

4 

5 

6 

7)) 

(5 . 22e-8 

3 . 19e-10 

9.87e-05 

3 . 92e-10) ) 

(((3) 

(1 

2 

4 

5 

6 

7)) 

( 1 . 04e-8 

6.35e-ll 

0.001096 

7.8e-ll)) 

(((4) 

(1 

2 

3 

5 

6 

7)) 

(9 . 35e-S 

6 . 28e-6 

9 . 88e-6 

7 . 72e-6) ) 

(((5) 

(1 

2 

3 

4 

6 

7)) 

(4.43e-5 

7 . 3e-7 

l.le-10 

1 . 95e-8) ) 

(((6) 

(1 

2 

3 

4 

5 

7)) 

(1.04e-9 

2.71#-7 

1. le-10 

1 . 59e-9) ) 

(((7) 

(1 

2 

3 

4 

5 

6)) 

(8 . 5e-6 

8 . 76e-8 

l.le-9 

7 . 72e-6) ) 

(((1 

2) 

(3 

4 

5 

6 

7)) 

(5 . 21e-5 

3 . 19e-7 

9 . 88e-8 

3 . 91e-7) ) 

(((1 

3) 

(2 

4 

5 

6 

7)) 

( 1 . 04e-5 

8 . 35e-8 

1 . le-6 

7 . 79e-8) ) 

(((1 

4) 

(2 

3 

5 

6 

7)) 

(0.093383 

0.006278 

9 . 89e-9 

0.007707)) 

(((1 

5) 

(2 

3 

4 

6 

7)) 

(0.044234 

0.000729 

l.le-13 

1 . 95e-05)) 

(((1 

6) 

(2 

3 

4 

5 

7)) 

(1 , C4e-08 

0.00027 

1 . 1#-13 

1 .59e-06) ) 

(((1 

7) 

(2 

3 

4 

5 

8)) 

(0.008489 

8 . 75e-05 

l.le-12 

0.007707)) 

(((2 

3) 

(1 

4 

5 

6 

7)) 

(S.22«-ll 

3 . 19e-13 

0.098584 

3 . 92e-13) ) 

(((2 

4) 

(1 

3 

5 

6 

7)) 

(4.7e-7 

3. 16e-8 

0.000888 

3.88e-08)) 

(((2 

5) 

(1 

3 

4 

6 

7)) 

(2.23e-7 

3 . 67e-9 

9 . 87e-9 

9.79e-ll)) 

(((2 

6) 

(1 

3 

4 

5 

7)) 

(6 . 22e-12 

1 . 36e-9 

9.87e-9 

7.99e-12)) 

(((2 

7) 

(1 

3 

4 

5 

8)) 

(4.27e-8 

4.4e-10 

9 . 88e-8 

3 . 88e-8) ) 

(((3 

4) 

(1 

2 

5 

6 

7)) 

( 1 . 7e-7 

2 . 84e-9 

0.009877 

7 . 57e-7) ) 

(((3 

5) 

(1 

2 

4 

6 

7)) 

(4.43e-8 

7. 3e-10 

l.le-7 

1 ,95e-ll)) 

(((3 

6) 

(1 

2 

4 

5 

7)) 

(1.04e-12 

2 . 71e-10 

l.le-7 

1 . 59e-12) ) 

(((3 

7) 

(1 

2 

4 

5 

8)) 

(8 . 51e-9 

8 . 77e-l 1 

l.le-6 

7 . 72e-9) ) 

(((4 

5) 

(1 

2 

3 

6 

7)) 

(0.000399 

7 . 22e-5 

9 . 88e-10 

1 . 93e-6) ) 

(((4 

6) 

(1 

2 

3 

5 

7)) 

(9 . 36e-9 

2 . 68e-6 

9 . 88e-10 

1 . 67e-7) ) 

(((4 

7) 

(1 

2 

3 

5 

8)) 

(7 . 65e-5 

8 . 68e-6 

9 . 89e-9 

0.000764)) 
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(((5 

6) 

(1 

2 

3 

4 

7)) 

(4 . 43e-9 

3.116-6 

1.16-14 

3 . 98e-10) ) 

(((5 

7) 

(1 

2 

3 

4 

8)) 

(3 . 62e-5 

1.016-6 

1. le-13 

1 . 93e-6) ) 

(((6 

(1 

2 

3 

4 

5)) 

(8 . 56-10 

3.746-7 

1. le-13 

1.57e-7)) 

(((1 

2 

3) 

(4 

5 

6 

7)) 

(5 . 226-8 

3.196-10 

9.876-5 

3.926-10)) 

(((1 

2 

4) 

(3 

5 

6 

7)) 

(0.000469 

3.156-5 

8.896-7 

3.876-5)) 

(((1 

2 

5) 

(3 

4 

6 

7)) 

(0.000222 

3.666-6 

9.886-12 

9.786-8)) 

(((1 

2 

6) 

(3 

4 

5 

7)) 

(5 . 216-9 

1.366-6 

9.886-12 

7.986-9)) 

(((1 

2 

7) 

(3 

4 

S 

6)) 

(4 . 276-5 

4.46-7 

9.896-11 

3.876-5)) 

(((1 

3 

4) 

(2 

5 

6 

7)) 

(9 . 356-5 

6.286-6 

9.886-6 

7.726-6)) 

(((1 

3 

5) 

(2 

4 

6 

7)) 

(4.43e-5 

7.36-7 

1.  le-10 

1.956-8)) 

(((1 

3 

6) 

(2 

4 

5 

7)) 

(1.046-9 

2.716-7 

1.16-10 

1.596-9)) 

(((1 

3 

7) 

(2 

4 

S 

6)) 

(8 . Se-6 

8.766-8 

l.le-9 

7.726-6)) 

(((1 

4 

5) 

(2 

3 

6 

7)) 

(0.398108 

0.072174 

9.89e-13 

0.001927)) 

(((1 

4 

6) 

(2 

3 

5 

7)) 

(9 . 346-6 

0.026766 

9.896-13 

0.000157)) 

(((1 

4 

7) 

(2 

3 

5 

6)) 

(0.076405 

0.008667 

9.96-12 

0.763028)) 

(((1 

5 

6) 

(2 

3 

4 

7)) 

(4. 426-6 

0.003108 

1.16-17 

3 . 97e-7) ) 

(((1 

5 

7) 

(2 

3 

4 

6)) 

(0.036192 

0  001007 

1.16-16 

0.001927)) 

(((1 

6 

7) 

(2 

3 

4 

5)) 

(8 . 496-7 

0.000373 

1.16-16 

0.000157)) 

(((2 

3 

4) 

(1 

5 

6 

7)) 

(4 . 76-10 

3. 16e-ll 

0.887255 

3.886-11)) 

(((2 

3 

5) 

(1 

4 

6 

7)) 

(2 . 236-10 

3.676-12 

9.866-6 

9.86-14)) 

(((2 

3 

6) 

(1 

4 

5 

7)) 

(5.236-15 

1.366-12 

9.866-8 

8.06-15)) 

(  (  (  2 

O 

7) 

/ 

V.  * 

-* 

5 

8)  ) 

(4.276-11 

4.416-13 

9.876-5 

3 . 88e-l 1 ) ) 

(((2 

4 

S) 

(1 

3 

6 

7)) 

(2 . 0«-6 

3.836-7 

8.886-8 

9.696-9)) 

(((2 

4 

6) 

Cl 

3 

5 

7)) 

(4.76-11 

1.3S6-7 

8.886-8 

7.916-10)) 

(((2 

4 

7) 

(1 

3 

5 

6)) 

(3 . 84e-7 

4.386-8 

8.896-7 

3 . 84e-6) ) 

(((2 

5 

8) 

(1 

3 

4 

7)) 

(2.236-11 

1.586-8 

9.876-13 

2.06-12)) 

(((2 

5 

7) 

(1 

3 

4 

8)) 

(1.826-7 

5.086-9 

9.886-12 

9 . 69e-9) ) 

(((2 

6 

7) 

(1 

3 

4 

S)) 

(4.276-12 

1.886-9 

9.886-12 

7.916-10)) 

(((3 

4 

5) 

(1 

2 

6 

7)  J 

(3.996-7 

7.236-8 

9.876-7 

1.936-9)) 

(((3 

4 

6) 

(1 

2 

5 

7)) 

(9.386-12 

2.886-8 

9.876-7 

1.586-10)) 

(((3 

4 

7) 

(1 

2 

5 

6)) 

(7.686-8 

8.686-9 

9.886-8 

7.656-7)) 

(((3 

5 

6) 

(1 

2 

4 

7)) 

(4.436-12 

3.116-9 

1.16-11 

3.98a-13)) 

(((3 

5 

7) 

(1 

2 

4 

6)) 

(3.836-8 

1.016-9 

1.16-10 

1.936-9)) 

(((3 

6 

7) 

(1 

2 

4 

5)) 

(8.516-13 

3.746-10 

1.16-10 

1.586-10)) 

(((4 

5 

6) 

(1 

2 

3 

7)) 

(3.996-8 

0.000308 

9.886-14 

3.9*r6-8)) 

(((4 

S 

7) 

(1 

2 

3 

6)) 

(0.000326 

9.986-5 

9.896-13 

0 . 1/00191) ) 

(((4 

e 

7) 

(1 

2 

3 

5)) 

(7.656-9 

3.76-5 

9.896-13 

1.566-5)) 

(((5 

6 

7) 

(1 

2 

3 

4)) 

(3.826-9 

4.36-8 

1.16-17 

3.946-8)) 

(((1 

2 

3  4) 

(6 

6 

7)) 

(4.76-7 

3.186-8 

0.000888 

3.886-8)) 

(((1 

2 

3  5) 

(4 

8 

7)) 

(2.236-7 

3.876-9 

9.876-9 

9.796-11)) 

(((1 

2 

3  8) 

(4 

S 

7)) 

(5.226-12 

1.386-9 

9.876-9 

7.996-12)) 

(((1 

2 

3  7) 

(4 

5 

6)) 

(4.276-8 

4.46-10 

9.886-8 

3.886-8)) 

(((1 

2 

4  6) 

(3 

8 

7)) 

(0.002001 

0.000363 

8.896-11 

9.686-6)) 

(((1 

2 

4  6) 

(3 

6 

7)) 

(4.896-8 

0.000134 

8.896-11 

7.96-7)) 

(((1 

2 

4  7) 

(3 

5 

6)) 

(0.000384 

4.386-5 

8.96-10 

0.003834)) 

(((1 

2 

5  6) 

(3 

4 

7)) 

(2.226-8 

1.586-5 

9.886-18 

2.06-9)) 

(((1 

2 

5  7) 

(3 

4 

8)) 

(0.000182 

5.086-8 

9.896-15 

9.686-6)) 

(((1 

2 

6  7) 

(3 

4 

5)) 

(4.276-9 

1.886-8 

9.896-15 

7.96-7)) 

(((1 

3 

4  5) 

(2 

6 

7)) 

(0.000399 

7.226-5 

9.886-10 

1.936-6)) 

(((1 

3 

4  8) 

(2 

5 

7)) 

(9.356-9 

2.686-5 

9.886-10 

1.576-7)) 

(((1 

3 

4  7) 

(2 

5 

8)) 

(7.856-5 

8.886-8 

9.896-9 

0.000764)) 
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(((1 

3 

5 

6) 

(2  4 

7)) 

(4 . 434-9 

3.119-0 

1.19-14 

3.984-10)) 

(((1 

3 

5 

7) 

(2  4 

0)) 

(3 . 62e-5 

1.019-6 

1.19-13 

1.934-6)) 

(((1 

3 

0 

7) 

(2  4 

5)) 

(8 . 54-10 

3.749-7 

1.14-13 

1.574-7)) 

(((1 

4 

5 

6) 

(2  3 

7)) 

(3 . 984-5 

0.307689 

9.894-17 

3.934-5)) 

(((1 

4 

5 

7) 

(2  3 

8)) 

(0.328725 

0.099669 

9.94-16 

0.190757)) 

(((1 

4 

6 

7) 

(2  3 

S)) 

(7 . 644-0 

0.036948 

9.94-16 

0.015572)) 

(((1 

5 

6 

7) 

(2  3 

4)) 

(3.624-6 

0.004292 

1.14-20 

3.934-5)) 

(((2 

3 

4 

5) 

(1  6 

7)) 

(2 . 04-9 

3.639-10 

8.874-5 

9.74-12)) 

(((2 

3 

4 

6) 

(1  5 

7)) 

(4 . 74-14 

1.359-10 

8.874-5 

7.924-13)) 

(((2 

3 

4 

7) 

(1  5 

8)) 

(3 . 854-10 

4.309-11 

0.000888 

3.844-09)) 

(((2 

3 

5 

6) 

(1  4 

7)) 

(2 . 234-14 

1.569-11 

9.864-10 

2.04-15)) 

(((2 

3 

5 

7) 

(1  4 

0)) 

( 1 . 82a- 10 

5.079-12 

9.874-9 

9.74-12)) 

(((2 

3 

S 

7) 

(1  4 

5)) 

(4.289-15 

1.889-12 

9.874-9 

7.924-13)) 

(((2 

4 

S 

6) 

(1  3 

7)) 

(2.09-10 

1.559-6 

8.889-12 

1.984-10)) 

(((2 

4 

5 

7) 

(1  3 

0)) 

(1.849-8 

5.019-7 

8.894-11 

9.64-7)) 

(((2 

4 

6 

7) 

(1  3 

5)) 

(3.849-11 

1.809-7 

8.894-11 

7.834-8)) 

(((2 

5 

6 

7) 

(1  3 

4)) 

(1.829-11 

2.169-8 

9.884-16 

1.984-10)) 

(((3 

4 

5 

6) 

(1  2 

7)) 

(3.999-11 

3.089-7 

9.874-11 

3.944-11)) 

(((3 

4 

5 

7) 

(1  2 

8)) 

(3.209-7 

9.999-8 

9.884-10 

1.914-7)) 

(((3 

4 

6 

7) 

(1  2 

5)) 

(7.809-12 

3.79-8 

9.884-10 

1.564-8)) 

(((3 

S 

6 

7) 

(1  2 

4)) 

(3.839-12 

4.39-9 

1.19-14 

3.944-11)) 

(((4 

5 

6 

7) 

(1  2 

3)) 

(3.209-8 

0.000425 

9.894-17 

3.94-6)) 

(((1 

2 

3 

4 

5)  (6 

7)) 

(2.09-6 

3.639-7 

8.884-8 

9.694-9)) 

(((1 

2 

3 

4 

6)  (5 

7)) 

(4.79-11 

1.359-7 

8.884-8 

7.914-10)) 

(((1 

2 

3 

4 

7)  (6 

6)) 

(3.849-7 

4.369-8 

8.894-7 

3.844-6)) 

(((1 

2 

3 

5 

6)  (4 

7)) 

(2.239-11 

1.569-8 

9.879-13 

2.04-12)) 

(((1 

2 

3 

5 

7)  (4 

8)) 

(1.829-7 

5 . 064-9 

9.884-12 

9.694-9)) 

(((1 

2 

3 

6 

7)  (4 

8)) 

(4.279-12 

1.884-9 

9.884-12 

7.914-10)) 

(((1 

2 

4 

5 

6)  (3 

7)) 

(2.09-7 

0.001546 

8.894-16 

1.984-7)) 

(((1 

2 

4 

6 

7)  (3 

6)) 

(0.001837 

0.000601 

8.94-14 

0.000959)) 

(((1 

2 

4 

6 

7)  (3 

5)) 

(3.849-8 

0.000188 

8.o«-l4 

7.834-5)) 

(((1 

2 

5 

6 

7)  (3 

4)) 

(4.049-12 

3.014-6 

9.894-20 

3.994-9)) 

(((1 

3 

4 

5 

6)  (2 

7)) 

(3.999-8 

0.000308 

9.884-14 

3.944-8)) 

(((1 

3 

4 

5 

7)  (2 

8)) 

(0.000326 

9 . 984-5 

9.894-13 

0.000191)) 

(((1 

3 

4 

6 

7)  (2 

6)) 

(7.669-9 

3.74-6 

9.894-13 

1.564-5)) 

(((1 

3 

5 

6 

7)  (2 

4)) 

(3.629-9 

4.39-0 

1.14-17 

3.944-8)) 

(((1 

4 

6 

6 

7)  (2 

3)) 

(3.269-6 

0.424903 

9.94-20 

0.003893)) 

(((2 

3 

4 

8 

6)  (1 

7)) 

(2.09-13 

1.554-9 

8.874-9 

1.984-13)) 

( (  (2 

3 

4 

8 

7)  (1 

8)) 

(1.649-9 

5.024-10 

8.884-8 

9.04-10)) 

(((2 

3 

4 

6 

7)  (1 

6)) 

(3.869-14 

1.264-10 

8.884-8 

7.844-11)) 

(((2 

3 

5 

6  7)  (1 

4)) 

(1.829-14 

2.164-11 

9.874-13 

1.984-13)) 

( ( (2 

4 

8 

6 

7)  (1 

3)) 

(1.649-10 

2.149-6 

8.894-15 

1.904-8)) 

(((3 

4 

8 

8 

7)  (1 

2)) 

(3.209-11 

4.209-7 

9.884-14 

3.94-9)) 

(((1 

2 

3 

4 

8  8) 

(7)) 

(2.09-10 

1.554-6 

8.884-12 

1.984-10)) 

(((1 

2 

3 

4 

8  7) 

(8)) 

(1.649-6 

5.014-7 

8.894-11 

9.64-7)) 

(((1 

2 

3 

4 

8  7) 

(8)) 

(3.849-11 

1.864-7 

8.894-11 

7.834-8)) 

(((1 

2 

3 

5 

8  7) 

(4)) 

(1.829-11 

2.164-8 

9.884-16 

1.984-10)) 

(((1 

2 

4 

S 

8  7) 

(3)) 

(1.849-7 

0.002135 

8.94-18 

1.964-6)) 

(((1 

3 

4 

S 

0  7) 

(2)) 

(3.289-8 

0.000428 

9.894-17 

3.94-6)) 

(((2 

3 

4 

8 

6  7) 

(l)) 

(1.849-13 

2.144-9 

8.884-12 

1.964-11)) 

(((1 

2 

3 

4 

8  6  7)0) 

(1.849-10 

2.149-6 

8.894-15 

1.964-8))  )) 

D-3 


D-4 


Appendix  E.  Quattro  Data 


The  following  data  was  generated  using  the  commercial  software  spreadsheet 
Quattro7  u .  The  P(  E|H)  matrix  and  the  P ( - E | H )  matrix  were  developed  from  inputs 
provided  by  the  expert.  The  P(  { E } | H )  is  a  combination  of  both  the  P(  E|H)  and  Pi  - 
E  H  i  matrices  and  is  the  knowdedge  base  accessed  by  the  decision-analytic  reasoner. 


P(EIH)  P(-EIH) 


1 

0.999 

0.999 

0.001 

0.999 

0.001 

0.001 

0.999 

0 .001 

2 

0.005 

0.005 

0.989 

0.005 

0.995 

0.995 

0.011 

0.995 

3 

0.001 

0  001 

0.999 

0.001 

0.999 

0.999 

0.001 

0.999 

4 

0.9 

0.99 

0.9 

0.99 

0.1 

0.01 

0.1 

0.01 

S 

0.81 

0.92 

0.0001 

0.2 

0.19 

0.08 

0.9999 

0.3 

6 

0.0001 

0.81 

0.0001 

0.02 

0.9999 

0.19 

0.9999 

0.98 

7 

C.45 

0.58 

0.001 

0.99 

0.55 

0.42 

0.999 

0.01 

12  0.004995 

0.004995 

0.000989 

0 . 004996 

0.000995 

0.000995 

0.010989 

0.000995 

13  0.000999 

0.000999 

0.000999 

0.000999 

0.000999 

0.000999 

0.000999 

0.000999 

14 

0.8991 

0.98901 

0 . 0009 

0.98901 

0.0001 

IE-05 

0.0999 

IE-05 

15 

0.80919 

0.91908 

IE-07 

0.1998 

0.00019 

8E-05 

0.9989 

0.0008 

16  ! 

3.99E-05 

0.80919 

IE-07 

0.01998 

0.001 

0.00019 

0.9989 

0.00098 

17 

0.44955 

0.57942 

IE-08 

0.98901 

0.00055 

0.00042 

0.998001 

1E-0S 

23 

5E-06 

5E-08 

0.988011 

5E-06 

0.994005 

0.994005 

1 . IE-05 

0.994005 

24 

0.0045 

0.00496 

0.8901 

0.00496 

0.0995 

0.00995 

0.0011 

0.00995 

25 

0  0040-5 

0.0046 

9.89E-05 

0.001 

0. 18905 

0.0796 

0.010999 

0.796 

26 

5E-07 

0.00405 

9.89E-05 

0.0001 

0 . 9949 

0.18905 

0.010999 

0.9751 

27 

0.00225 

0.0029 

0.000989 

0.00496 

0.54725 

0.4179 

0.010989 

0.00995 

34 

0.0009 

0.00099 

0.8991 

0.00099 

0.0999 

0.00999 

0.0001 

0 . 0C999 

35 

0.00081 

0.00092 

9.99E-06 

0.0002 

0.18981 

0.07992 

0.001 

0.7992 

36 

IE-07 

0.00081 

9 . 99E-0o 

2E-06 

0.9989 

0.18981 

0.0C1 

0 . 97902 

37 

0.00045 

0.00068 

0.000999 

0.00099 

0.54945 

0.41958 

0.000999 

0.0C999 

45 

0.729 

0.9108 

9E-0S 

0.198 

0.019 

0.0008 

0.09999 

0.008 

46 

9E-06 

0.8019 

9E-05 

0.0198 

0.09999 

0.0019 

0.09999 

0.0098 

47 

0 . 405 

0.5742 

0.0009 

0.9801 

0.055 

0.0042 

0.0999 

0.0001 

56 

8.  IE-06 

0.7462 

IE-08 

0.004 

0.189981 

0.0152 

0.9998 

0.784 

57 

0.3645 

0.5336 

IE-07 

0.198 

0.1045 

0.0336 

0.9989 

0.008 

67 

4  .  SE-05 

0.4698 

IE-07 

0.0198 

0.549945 

0.0798 

0.9989 

0 . 0098 

123 

5E-06 

5E-06 

0.000988 

5E-06 

0.000994 

0.000994 

1.1E-05 

0.000994 

124  0.004496 

0.004945 

0.00089 

0.004945 

9.95E-05 

9.95E-06 

0.001099 

9.95E-06 

125  0.004046 

0.004595 

9.89E-08 

0.000999 

0.000189 

7. 96E-05 

0.010988 

0.000796 

126 

5E-07 

0 . 004046 

9.89E-08 

9.99E-06 

0.000995 

0.000189 

0.010988 

0.000975 

127  0.002248 

0.002897 

9.89E-07 

0 . 004945 

0.000547 

0.000418 

0.010978 

9.9SE-06 

134  0.000899 

0.000989 

0.000899 

0.000989 

9.99E-05 

9.99E-06 

9.99E-05 

9.99E-06 

E-l 


135  0.000809  0.000919  9.99E-08  0.0002 

136  9.99E-08  0.000809  9.99E-08  2E-0S 

137  0 . 00045  0.000579  9.99E-07  0.000989 

145  0.728271  0.909889  9E-08  0.197802 

146  8.99E-05  0.801098  9E-08  0.01978 

147  0.404595  0.S73C26  9E-07  0.97912 

156  8.09E-05  0.744455  IE-11  0.003996 

157  0.364136  0.533066  IE-10  0.197802 

167  4.5E-05  0.46933  IE-10  0.01978 

234  4.5E-06  4.95E-06  0.88921  4 . 95E-08 

235  4.0SE-06  4.6E-06  9 . 88E-05  IE-06 

236  SE-10  4.0SE-06  9.88E-05  IE-07 

237  2.25E-06  2.9E-06  0.000988  4.9SE-06 

245  0.003645  0.004554  8.9E-0S  0.00099 

246  4.5E-07  0.00401  8.9E-05  9 . 9E-05 

247  0.002025  0.002871  0.00089  0.004901 

256  4.05E-07  0.003726  9 . 89E-09  2E-0S 

257  0.001823  0.002668  9.89E-08  0.00099 

267  2.25E-07  0.002349  9.89E-08  9.9E-06 

345  0.000729  0.000911  8.99E-05  0.000198 

346  9E-08  0.000802  8 . 99E-0S  1 . 98E-06 

347  0.000405  0.000574  0 . 000899  0.00098 

356  8. IE-08  0.000745  9. 99E-09  4E-06 

357  0.000365  0.000534  9 . 99E-08  0.000198 

367  4.5E-08  0.00047  9 . 99E-08  1 . 98E-0S 

456  7.29E-05  0.737748  9E-09  0.00396 

457  0.32805  0.528264  9E-08  0.19802 

467  4 . 05E-05  0.465102  9E-08  0.019602 

567  3.65E-05  0.  <132216  lE-il  0.00396 

1234  4.5E-06  4.95E-06  0.000889  4 . 95E-06 

1235  4.05E-06  4.6E-06  9 . 88E-08  9 . 99E-07 

1236  5E-10  4.05E-06  9 . 88E-08  9.99E-08 

1237  2.25E-06  2 . 9E-06  9.88E-07  4.95E-06 

1246  0.003641  0.004549  8.9E-08  0.000989 

1246  4.5E-07  0.004005  8.9E-08  9 . 89E-05 

1247  0.002023  0.002868  8.9E-07  0.004896 

1256  4.05E-07  0.003722  9.89E-12  2E-06 

1257  0.001821  0.002665  9.89E-11  0.000989 

1267  2.25E-07  0.002347  9.89E-11  9.89E-05 

1345  0.000728  0.00091  8.99E-08  0.000198 

1346  8.99E-08  0.000801  8 . 99E-08  1.98E-05 

1347  0.000406  0.000574  8 . 99E-07  0.000979 

1356  8.09E-08  0.000744  9.99E-12  4E-06 

1357  0.000364  0.000533  9.99E-11  0.000198 

1367  4.5E-08  0.000469  9.99E-11  1 . 98E-05 

1456  7.28E-05  0.73701  9E-12  0.003966 

1457  0.327722  0.527736  9E-11  0.195824 

1467  4.05E-0S  0.464637  9E-11  0.019582 

1567  3.64E-05  0.431784  IE-14  0.003956 

2345  3.6SE-08  4 . 55E-06  8.89E-05  9  . ES-n? 

2346  4.5E-10  4.01E-06  8 . 89E-05  9 . 9E-08 


0.00019  7.99E-05  0.000999  0  001799 
0.000999  0.00019  0.000999  0.001979 

0.000549  0.00042  0.000998  9.99E-16 

1.9E-0S  8E-07  0.09989  3E-06 

IE-04  1.9E-06  0.09989  9.8E-16 

5.5E-05  4.2E-06  0.0998  1E-C7 

0.00019  1.52E-05  0.9988  0.CC0734 

0.000105  3.36E-05  0.997901  3E-16 

0.00055  7.98E-05  0.997901  9.3E-16 

0.0994  0.00994  1 . IE-06  0.01994 

0.188861  0.07952  1 . IE-05  0 . 795214 

0.993906  0.188861  1 . 1E-0S  1 . 374 ; 25 

0.5467030.417482  1.1E-CS  0.00994 

0.018905  0.000796  0.0011  0.00736 

0.09949  0.001891  0.0011  0.009751 

0.054725  0.004179  0.001099  9.SoE-15 
0.189031  0.015124  0.010998  0.731:3 

0.103977  0.033432  0.010988  O.CC’96 

0.547196  0.079401  0.010988  O.CC9'5l 
0.018981  0.000799  IE-04  0. 007992 

0.09989  0.001898  IE-04  0.00979 

0.054945  0.004196  9.99E-05  9.99E-C5 
0.189791  0.015185  0.001  0.783216 

0.104395  0.033566  0.000999  0.007992 
0.549395  0.07972  0.000999  0  119TS 

0.018998  0.0001S2  0.09998  0.007=4 

0.01045  0.000336  0.09989  SE-15 

0.054994  0.000798  0.09989  9.8E-15 

0.104490.006384  0.9988  0.C0'34 

9.94E-05  9.94E-06  1 . IE-06  9.94E-06 

0.000189  7.95E-05  1.1E-05  0.C0C'95 

0.000994  0.000189  1 . 1E-0S  0  .  G0C974 

0.000547  0.000417  1 . IE-05  9.94E-06 

1.89E-05  7.96E-07  0.001099  7.96E-16 
9.95E-05  1.89E-06  0.001099  9.75E-C6 
5.47E-05  4.18E-06  0.001098  9.9SE-08 
0.000189  1.51E-05  0  010987  0.0017S 

0.000104  3.34E-05  0.010977  7.962-16 
0.0C0547  7.94E-05  0.010977  9  75E-16 
1.9E-05  7.99E-07  9 . 99E-05  7  99E-16 
9.99E-05  1.9E-06  9.99E-05  9.79E-16 

5.49E-0S  4.2E-06  9 . 98E-05  9.99E-C8 

0.00019  1.52E-05  0.000999  0.0CC'83 
0.000104  3.36E-05  0.000998  7  99E-C6 
0.000649  7.97E-05  0.000998  9  79E-C6 
1.9E-05  1.52E-07  0.09988  7.84E-16 

1 .05E-0S  3.36E-07  0.09979  8E-C8 

5.5E-06  7.98E-07  0.09979  9.8E-18 

0.000104  6.38E-06  0.997801  7.84E-16 
0.0188560.000795  1 . IE-06  0 . C07952 

0.099391  0.001889  1 . IE-06  0  009741 


E-2 


2347 

2.03E-06 

2.87E-06 

0.000889 

4.9E-06 

0.05467 

0.004175 

1 .  IE-06 

9  94E-05 

235S 

4.05E-10 

3.73E-06 

9.88E-09 

2E-08 

0.183842 

0.015109 

1  .  1E-0S 

0.7793 

2357 

1 .82E-06 

2.67E-06 

9.88E-08 

9.9E-07 

0.103874 

0.033399 

1. IE-05 

0.007952 

2367 

2.25E-10 

2.3SE-06 

9.88E-08 

9.9E-08 

0.546648 

0.079322 

1 . IE-05 

0.009741 

24S6 

3.65E-07 

0.003689 

8.9E-09 

1 .98E-05 

0.018903 

0.000151 

0.0011 

0.007801 

2457 

0.00164 

0.002641 

8.9E-08 

0.00098 

0.010398 

0.000334 

0.001099 

7.96E-05 

2467 

2.03E-07 

0.002326 

8.9E-08 

9.8E-05 

0.05472 

0.000794 

0.001C99 

9.75E-05 

2567 

1 .82E-07 

0.002161 

9.89E-12 

1.98E-0S 

0.103967 

0.006352 

0.010987 

0.007801 

3456 

7.29E-08 

0.000738 

8.99E-09 

3.96E-06 

0.018979 

0.000152 

IE-04 

0.007832 

3457 

0.000328 

0.000528 

8.99E-08 

0.000196 

0.01044 

0.00''336 

9.99E-05 

7. 99E-C5 

3467 

4.05E-08 

0.000465 

8.99E-08 

1 .96E-0S 

0.05494 

0.000797 

9.99E-05 

9.79E-05 

3567 

3.65E-08 

0.000432 

9.99E-12 

3.96E-06 

0.104385 

0.006378 

0.000999 

0.007832 

4567 

3.28E-05 

0.427894 

9E-12 

0.00392 

0.010449 

6.38E-05 

0.09988 

7.34E-0S 

12345 

3.64E-06 

4.55E-06 

8.89E-08 

9.89E-07 

1.89E-05 

7.95E-07 

1 . IE-06 

7.95E-06 

22346 

4.5E-10 

4.01E-06 

8.89E-08 

9 . 89E-08 

9 . 94E-05 

1.89E-06 

1 . IE-06 

9.74E-06 

12347 

2.02E-06 

2.87E-06 

8.89E-07 

4.9E-06 

5.47E-0S 

4. 17E-06 

1 . IE-06 

9.94E-08 

12356 

4.05E-10 

3.72E-06 

9.88E-12 

2E-08 

0.000189 

1.51E-0S 

1 . IE-05 

0.000779 

12357 

1 .82E-06 

2.67E-06 

9.88E-11 

9.89E-07 

0.000104 

3.34E-05 

1 . IE-05 

7 . 95E-06 

12367 

2.25E-10 

2.3SE-06 

9.88E-11 

9.89E-08 

0.000547 

7.93E-05 

1 . 1E-0S 

9.74E-06 

12456 

3.64E-07 

0.003685 

8.9E-12 

1.98E-05 

1 .89E-0S 

1.51E-07 

0.001099 

7.8E-06 

12457 

0.001639 

0.002639 

8.9E-11 

0.000979 

1.04E-05 

3.34E-07 

0.001098 

7.96E-08 

12467 

2.02E-07 

0.002323 

8.9E-11 

9  79E-05 

S.47E-05 

7.94E-07 

0.001098 

9.75E-08 

12567 

4.05E-11 

0.003016 

9.89E-16 

4E-07 

0.000189 

2.87E-06 

0.010986 

0.000764 

13456 

7.28E-08 

0.000737 

8.99E-12 

3.96E-06 

1.9E-05 

1.S2E-07 

9.99E-05 

7.83E-06 

13457 

0.000328 

0.000528 

8.99E-11 

0.000196 

1 . 04E-05 

3.36E-07 

9.98E-05 

7 . 99E-08 

13467 

4.05E-08 

0.000466 

8.99E-11 

1 . 98E-0S 

5.49E-06 

7.97E-07 

9.98E-05 

9.79E-08 

13567 

3.64E-08 

0.000432 

9.99E-1S 

3 . 96E-06 

0.000104 

6.38E-06 

0.000998 

7.83E-06 

14567 

3.28E-05 

0.427466 

9E-1S 

0.003916 

1 .04E-05 

6.38E-08 

0.09978 

7.84E-08 

23456 

3.65E-10 

3.69E-06 

8.89E-09 

1.98E-08 

0.018884 

0.000151 

1.1E-06 

0.007793 

23457 

1 .64E-06 

2 . 64E-06 

8.89E-08 

9.8E-07 

0.010387 

0.000334 

1. IE-06 

7.95E-05 

23467 

2.03E-10 

2.33E-06 

8.89E-08 

9.8E-08 

0.054665 

0.000793 

1 . IE-06 

9.74E-05 

23567 

1.82E-10 

2. 16E-06 

9.88E-12 

1 . 98E-08 

0. 103863 

0.006346 

1.1E-05 

0.007793 

24S67 

1.64E-07 

0.002139 

8.9E-12 

1 . 96E-05 

0.010397 

6.35E-05 

0.001099 

7.8E-05 

34567 

3.28E-08 

0.000428 

8.99E-12 

3 . 92E-06 

0.010439 

6.38E-05 

9.99E-05 

7.83E-05 

123456 

3.64E-10 

3.69E-06 

8.89E-12 

1.98E-08 

1 . 89E-06 

1.51E-07 

1.1E-06 

7.79E-06 

123457 

1.64E-06 

2.64E-06 

8.89E-11 

9.79E-07 

1.04E-05 

3.34E-07 

1 . IE-06 

7.95E-08 

123467 

2.02E-10 

2.32E-06 

8.89E-11 

9.79E-08 

5.47E-05 

7.93E-07 

1. IE-06 

9.74E-08 

123567 

1.32E-10 

2. 16E-06 

9.88E-15 

1.98E-08 

0.000104 

6.35E-06 

1 . IE-05 

7.79E-06 

124667 

1.64E-07 

0.002137 

8.9E-15 

1.96E-05 

1.04E-0S 

6.35E-08 

0.001098 

7.8E-08 

134567 

3.28E-08 

0.000427 

8.99E-15 

3.92E-06 

1 .04E-05 

6.38E-08 

9.98E-05 

7.83E-08 

234567 

1.64E-10 

2. 14E-06 

8.89E-12 

1 .96E-08 

0.010386 

6.35E-05 

1. IE-06 

7. 79E-05 

1234567 

1.64E-10 

2. 14E-06 

8.89E-15 

1.96E-08 

1.04E-06 

6.35E-08 

1 . IE-06 

7 . 79E-08 

E-3 


P({E}|H) 


1.234S67  0.010376  6.34E-06  1.1E-09  7.79E-05 

2,134567  5.22E-08  3.19E-10  9.87E-06  3.92E-10 
3,124567  1.04E-08  6.35E-11  0.001096  7.8E-11 

4,123567  9.3SE-05  6.28E-06  9.88E-06  7.72E-06 
5, 123467  4.43E-0S  7.3E-07  1 . IE-10  1 . 95E-08 

6,123457  1.04E-09  2.71E-07  1.1E-10  1.59E-09 

7,123456  8.5E-06  8.76E-08  1.1E-09  7.72E-06 

12,34567  5.21E-05  3.19E-07  9.88E-08  3.91F.-07 
13,24667  1.04E-05  6.35E-08  1.1E-06  7.79E-08 

14,23567  0.093383  0.006276  9.89E-09  0.007707 
15,23467  0.044234  0.000729  1.1E-13  1.95E-06 

16,23457  1.04E-06  0.00027  1.1E-13  1.59E-06 

17,23466  0.008489  8.75E-05  1.1E-12  0.007707 

23,14567  5.22E-11  3.19E-13  0.098584  3.92E-13 
24,13567  4.7E-07  3.16E-08  0.000888  3.88E-08 

25,13467  2.23E-07  3.67E-09  9.87E-09  9.79E-11 
26,13457  5.22E-12  1.36E-09  9.87E-09  7.99E-12 
27,13456  4.27E-08  4.4E-10  9.88E-08  3.88E-08 
34,12567  1.72-07  2.84E-09  0.009877  7.57E-07 

35,12467  4.43E-08  7.3E-10  1.1E-07  1.96E-11 

38,12457  1.04E-12  2.71E-10  1.1E-07  1.S9E-12 

37,12466  8.61E-09  8.77E-11  1 . IE-06  7.72E-09 

45,12367  0.000399  7.22E-OS  9.88E-10  1.93E-06 
46,12357  9.3SE-09  2.68E-0S  9.88E-10  1.57E-07 
47,12356  7.65E-06  8.68E-06  9.89E-09  0.000764 

56.12347  4.43E-09  3.11E-06  1.1E-14  3.98E-10 

57.12348  3.62E-05  1.01E-06  1.1E-13  1.93E-06 

67,12345  8.5E-10  3.74E-07  1.1E-13  1.57E-07 

123,4567  5.22E-08  3.19E-10  9.87E-05  3.92E-10 
124,3567  0.000469  3.15E-0S  8.89E-07  3.87E-05 

125.3467  0.000222  3.68E-06  9.88E-12  9.78E-08 

126.3467  5.21E-09  1.36E-06  9.88E-12  7.98E-09 
127,3456  4.27E-05  4.4E-07  9.89E-11  3.87E-05 
134,2567  9.3SE-0S  8.28E-08  9.88E-06  7.72E-06 

135.2467  4.43E-05  7.3E-07  1.1E-10  1.96E-08 

136.2467  1.04E-09  2.71E-07  1.1E-10  1.S9E-09 

137,2466  8.5E-06  8.76E-08  1.1E-09  7.72E-06 

146.2367  0.398108  0.072174  9.89E-13  0.001927 

148.2367  9.34E-06  0.028756  9.89E-13  0.000157 

147.2368  0.078405  0.008687  9.9E-12  0  763028 

158,2347  4.42E-06  0.003108  1.1E-17  3.97F.-07 

157,2346  0.038192  0.001007  1.1E-1S  0.001927 

167,2345  8.49E-07  0.000373  l.iE-16  0.000157 

234,1587  4.7E-10  3.18E-11  0.887255  3.88E-11 

235,1487  2.23E-10  3.67E-12  9.86E-06  9.8E-14 

236,1457  5.23E-15  1  38E-12  9.86E-06  8E-15 

237,1456  4.27E-11  4.41E-13  6.87E-05  3.88E-11 
245,1367  2E-08  3.63E-07  8.88E-08  9.69E-09 

248,1357  4.7E-11  1.35E-07  8.88E-08  7.91E-10 


E-4 


247,1356 

256,1347 

257,1346 

267,1345 

345,1267 

346,1257 

347,1256 

356,1247 

357,1246 

367,1245 

456,1237 

457,1236 

467,1235 

567,1234 

1234,567 

1235.456 

1236.457 
1237,456 
1245,367 
1246,357 
1247,356 
1256,347 

1257.346 

1267.346 
1345,267 
1346,257 
1347,256 
1356,247 
1357,246 
1367.245 
1456  237 

1457.236 

1467.236 
1567,234 
2345,167 
2346,157 
2347,166 
2358,147 

2357.146 

2367.146 
2466,137 
2467,136 
2467,136 
2667,134 

3458.127 

3457.128 
3467,125 
3587,124 
4567,123 
12346,67 
12348,57 


3.84E-07 
2.23E-11 
1.82E-07 
4.27E-12 
3.99E-07 
9.36E-12 
7.66E-08 
4.43E-12 
3.63E-08 
8.S1E-13 
3.99E-08 
0.000326 
7.65E-09 
3.62E-09 
4.7E-07 
2.23E-07 
5.22E-12 
4.27E-08 
0.002001 
4.69E-08 
0.000384 
2.22E-08 
0.000182 
4.27E-09 
0.000399 
9 . 35E-09 
7.65E-05 
4.43E-09 
3.62E-06 
8.5E-10 
3.98E-05 
0.325725 
7 . 64E-06 
3.62E-06 
2E-09 
4.7E-14 
3.86E-10 
2.23E-14 
1 . 82E-10 
4.28E-15 
2E-10 
1.64E-06 
3.84E-U 
1.82E-11 
3.99E-11 
3.28E-07 
7.66E-12 
3.63E-12 
3 . 26E-08 
2E-06 
4.7E-11 


'  36E-08 
1.56E-08 
5.06E-09 
1.88E-09 
7.23E-08 
2.68E-08 
8 . 68E-09 
3.UE-09 
1.01E-09 
3.74E-10 
0.000308 
9.98E-05 
3.7E-05 
4.3E-06 
3.16E-08 
3.67E-09 
1 .36E-09 
4.4E-10 
0.000363 
0.000134 
4.36E-05 
1.S6E-06 
5.06E-06 
1.88E-06 
7.22E-05 
2.68E-06 
8 . 68E-06 
3.UE-08 
1.01E-08 
3 . 74E-07 
0.307689 
0.099669 
0.036948 
0 . 004292 
3.63E-10 
1.35E-10 
4.36E-11 
1.56E-U 
5.07E-12 
1.88E-12 
1.55E-06 
6.01E-07 
1.86E-07 
2.16E-08 
3 . 08E-07 
9 . 99E-08 
3.7E-08 
4.3E-09 
0.000425 
3 . 63E-07 
1 . 35E-07 


8 . 89E-07 
9.87E-13 
9.88E-12 
9.88E-12 
9.87E-07 
9.87E-07 
9.88E-06 
1.1E-11 
1.1E-10 
1.1E-10 
9.88E-14 
9.89E-13 
9.89E-13 
1.1E-17 
0.000888 
9.87E-09 
9.87E-09 
9.88E-08 
8.89E-11 
8.89E-11 
8.9E-10 
9.88E-16 
9.89E-15 
9.89E-15 
9.88E-10 
9.88E-10 
9.89E-09 
1.1E-14 
1.1E-13 
1.1E-13 
9.89E-17 
9.9E-16 
9.9E-16 
1 . IE- 20 
8.87E-05 
8.87E-06 
0.000888 
9.86E-10 
9.87E-09 
9.87E-09 
8.88E-12 
8.89E-11 
8.89E-11 
9.88E-16 
9.87E-U 
9.88E-10 
9.88E-10 
1.1E-14 
9.89E-17 
8 . 88E-08 
8 . 88E-08 


3.84E-06 
2E-12 
9 . 69E-09 
7.91E-10 
1.93E-09 
1.58E-10 
7.65E-07 
3.98E-13 
1.93E-09 
1.S8E-10 
3.94E-08 
0.000191 
1.56E-0S 
3.94E-08 
3.88E-08 
9.79E-U 
7.99E-12 
3.88E-08 
9.68E-06 
7.9E-07 
0.003834 
2E-09 
9.68E-06 
7.9E-07 
1 . 93E-06 
1.57E-07 
0.000764 
3.98E-10 
1.93E-06 
1 . 57E-07 
3.93E-05 
0. 190757 
0.015572 
3.93E-05 
9.7E-12 
7.92E-13 
3 . 84E-09 
2E-15 
9.7E-12 
7.92E-13 
1.98E-10 
9.6E-07 
7.83E-08 
1.98E-10 
3.94E-U 
1.91E-07 
1 . 56E-08 
3 . 94E-11 
3 . 9E-06 
9.69E-09 
7.91E-10 
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12347,56 

12356,47 

12357.46 

12367.46 
12456,37 
12457,36 
12467,35 
12567,34 
13456,27 
13457,26 
13467,25 
13567,24 
14567,23 
23456,17 
23457,16 
23467,15 
23567,14 
24567,13 
34567,12 
123456,7 
123457,6 
123467,5 
123567,4 
124667,3 
134667,2 
234567 , 1 
1234567 


3.84E-07 
2.23E-11 
1 . 82E-07 
4.27E-12 
2E-07 
0.001637 
3.84E-08 
4.04E-12 
3.99E-08 
0.000326 
7 . 65E-09 
3.62E-09 
3.26E-05 
2E-13 
1 . 64E-09 
3.85E-14 
1.82E-14 
1.64E-10 
3.26E-11 
2E-10 
1.64E-06 
3.84E-11 
1.82E-11 
1 . 64E-07 
3.26E-08 
1.64E-13 
1.64E-10 


4.36E-08 
1.56E-08 
5.06E-09 
1.88E-09 
0.001546 
0.000501 
0.000188 
3.01E-05 
0.000308 
9.98E-05 
3.7E-05 
4.3E-06 
0.424903 
1 . 55E-09 
5.02E-10 
1.86E-10 
2.16E-11 
2.14E-06 
4.26E-07 
1.5SE-06 
5.01E-07 
1.86E-07 
2.16E-08 
0.002135 
0 . 000425 
2.14E-09 
2 . 14E-06 


8.89E-07 
9.87E-13 
9.88E-12 
9.88E-12 
8.89E-15 
8.9E-14 
8.9E-14 
9.89E-20 
9.88E-14 
9.89E-13 
9.89E-13 
1. IE- 17 
9.9E-20 
8.87E-09 
8.88E-08 
8.88E-08 
9.87E-13 
8.89E-1S 
9.88E-14 
8.88E-12 
8.89E-11 
8.89E-11 
9.88E-16 
8.9E-18 
9.89E-17 
8.88E-12 
8.89E-15 


3.84E-06 
2E-12 
9.69E-09 
7.91E-10 
1.98E-07 
0.000959 
7.83E-05 
3.99E-09 
3.94E-08 
0.000191 
1.56E-05 
3.94E-08 
0.003893 
1.98E-13 
9.6E-10 
7.84E-11 
1.98E-13 
1.96E-08 
3.9E-09 
1.98E-10 
9.6E-07 
7 . 83E-08 
1.98E-10 
1 . 96E-05 
3 . 9E-06 
1.96E-11 
1.96E-08 


E-6 


Appendix  F.  Computer  Output 


The  following  output  testing  was  done  in  order  to  determine  if  the  computer 
results  would  match  that  of  the  expert.  Neither  conflicting  evidence  information  nor 
uncertainty  assessments  were  incorporated  in  this  test.  The  responses  recommended 
by  the  DA  reasoner  are  determined  from  the  joint  likelihood  matrix  and  the  scaled 
utility  table.  After  the  recommended  action,  the  expert  expected  action  is  given. 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  1 ) (4  1) (7  1) ) ( (2  1) (3  1) (S  l)  (6  1))) 


14 . 9160660602193 
7.62023493 
47.7074220910387 
75. 3235620340039 


1 0 . 102468632262818  I 
10.0523485405700168  I 
10.327734604744941  I 
10.517448422422224  I 


Do  action  4  Expected  4 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  1) (5  1) (6  1) ) ( (2  1) (3  1) (4  1) (7  1))) 


0.00164951870500002  10.0177405542122384  I 
0.060703116705  10.662861304023074  I 
0.00626684711000004  10.0673891117473198  | 
0.02436162877  10.26200903001737  I 


Do  action  2  Expected  2 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8)) ((3  .6))) 

(((1  1) (8  1))((2  1) (3  1) (4  1) (5  1) (7  1))) 


2 . 2255475021483e-4 
0.00528882835 
6 . 6109466042966e-4 
0.00227466014004297 


10.026347394230459 

10.62609997426044 

10.0782644343703208 

10.269288197138781 


I 

I 

I 


Do  action  2  Expected  2 


F'l 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8 ) ) ( (3  .6))) 


(((1  1) (2  1) (3  1) ) ( (4  1) (5  1) (6  1) (7  1))) 


1.978629044814e-4 
1 ,005795e-8 
3.87S84931087e-4 
3.91027760286-5 


10.316803337336471  I 
1 1.61040399923114e-S  I 
10.620572107700108  I 
10.0626084509234294  | 


Do  action  3  Expected  3 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  - 8 ) ) ( ( 3  .6))) 

(((2  1) (3  1))((1  1) (4  1) (5  1) (6  1) (7  1))) 


0.192534557101804 
1 ,005795e-ll 
0.385069106062731 
0.038506910950556 


10.312500004362359  I 
ll.63249105313309e-ll  I 
10.624999995511378  I 
10.0625000001099384  I 


Do  action  3  Expected  3 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  1) ) ( (2  1) (3  1) (4  1) (5  1) (6  1) (7  1))) 


1.0141056196883 

0.0019988955 

0.4100160494966 

0.10943423222966 


10.660416431719646  I 
10.00130174156208326  I 
10.26701492536799  I 
10.0712669013502805  I 


Do  action  1  Expected 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  ■ 8) ) ( (3  .6))) 

(((1  1) (2  1) ) ( (3  1) (4  1) (5  1) (6  1) (7  1))) 


0.0050922216728 
1 . 0048186e-6 
0.0020591520098 
5 . 4952036928e-4 


10.6603890311154  I 
10.00130310728459894  I 
10.267042852422297  I 
10.0712650091777042  I 


Do  action  1  Expected  1 


F-2 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  1)  (3  1) )  ( (2  1)<4  DCS  1)  (6  1)(7  1))) 


0.0010185977966  10.65888228117107  I 
2 . 0008485e-6  10.00129425336316078  I 
4. 152503765e-4  10.268605642225739  I 
1.10099057«-4  10.0712178232400308  | 


Do  action  1  Expected  1 


F-3 


Appendix  G.  Computer  Output  Mixed  Data 


The  following  output  shows  the  results  of  mixing  conflicting  subsets  of  the 
various  Evidences.  Some  cases  also  show  changes  given  varying  belief  estimates. 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))(C3  .6))) 

f  C (1  1 ) (4  1) (6  1 ) ) ( (2  1) (3  1))) 


0.511754662901934  10.018878721949372  I 
15.7436534745  10.580786220348139  I 
2.71061319532387  1 0 . 099995010375947  I 
8.14146318208038  10.300340047326542  I 


Do  action  2 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  1) (4  1 ) (6  1 ) ) ( (2  1) (3  1) (5  1) (7  1))) 


0.0128961430019315  10.0156652592287429  I 
0.524077735  ! 0 . 6366100590557  I 
0.061817955763863  10.0760917775870532  I 
0.224440136440386  10.272632904128604  I 


Do  action  2 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  - 8) ) ( (3  .6))) 

(((1  1) (4  1) (7  1))((2  1) (3  1))) 


49.0024469306193  10.198464062459614  I 
20.63953836  ! 0 . C83E2 13793201551  I 
73 . 8466709663987  1 0 . 2990852750651 54  I 
103.419757907844  10.418858783155077  I 


Do  action  4 


G-l 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  1)(4  1X7  1) )  ( (2  1)  (3  1)<5  1X6  1))) 


14.9160660502193  10.102468532262818  I 
7.62023493  10.0623485405700168  I 
47.7074220910387  10.327734504744941  I 
75.3235620340039  10.517448422422224  I 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8 ) ) ( (3  .6))) 

( C  ( 1  1X4  1) )  ( (2  1))) 


97.3609568474787  10.292171302733066  I 
28.828195564896  10.0865107711177695  I 
94.5251347987174  10.283661260831941  I 
112.518155320848  10.337656665317224  I 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

((Cl  1X4  1) (3  1 ) )  ( (2  1))) 


0.0974301686403542  10.292227161699813  I 
0 . 028824975396  1 0 . 0884661945047594  I 
0.0945985527207083  10.283734184037681  I 
0.112551866900671  10.337582459767746  I 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8) ) ( (3  .6))) 

(((1  1) (4  1))((2  1) (3  1) (5  1) (6  1) (7  1))) 


9.19749405991617  10.60427767062763  I 
0.197829135  10.0129974238723671  I 
4.11172370863034  10.270141280692198  I 
1.71359504386303  10.112583824907804  I 


Do  action  1 


G-2 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  ,8))((3  .6))) 

(((1  1) ) ( (2  1) (3  1) (4  1) (5  l)(o  1) (7  1))) 


1 . 0141056196883  10.660416431719646  I 
0.0019988955  10.001301741562083263  I 
0.4100160494966  10.26701492536799  I 
0.10943423222966  10.0712669013502805  I 


Do  action  1 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8 ) ) ( ( 3  .6))) 

(((1  1 ) (5  1 ) (6  1) ) ( (2  1) (3  1))) 


0.335683196305  10.014802773053579  I 
14.490830310705  10.63900866891291  I 
1.68106598839  10.0741307238138238  I 
6.16946858809  10.272057834219687  I 


Do  action  2 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((2  1) (3  1))) 


1.93012435611677  10.312498215740031  I 
1.4659819182225e-4  I  2 . 37350890008769e-53  I 
3.85966878381156  10.624902637739005  I 
0.386493466611369  10.0625756114319628  I 


Do  action  3 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  8))((3  .6))) 

(((2  1) (3  1 ) ) ( (7  1))) 


1.92792667312466  10.312619717850631  I 
4. 155327770517e-5  1 6 . 73686024838715e-63  I 
3.86542633839863  1 0 . 62497C541274767  I 
0.385679344761487  10.062503005024354  I 


Do  action  3 


G-3 


Conflicting  Data 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  -8))((3  .6))) 


(('1  1)  (2  0(3  1 )  (6  1))((S  1)(7  1))) 


2 . 5886666855e-7 
2 . 6709129373Se-6 
7 . 0056884618e-7 
1 . 18231617602e-6 


I  0 . 053788636S56667S  I 
1 0 . 554975911214585  I 
10.145567767612467  | 
10.245667684616281  I 


Do  action  2 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 


(((1  1 )  (6  1)  (7  1 )  )  (  (2  0(3  1))) 


0.378580025040002 

9.3030500745 

2.064807827376 

5.564785565094 


10.0218690507470333  I 
10.537399917388498  I 
10.11927567270616  I 
10.321455359158319  I 


Do  action  2 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8) ) ( (3  .6))) 


(((1  1 ) (2  1) (3  1) (7  1) ) ( (6  1) (5  1))) 


8. 1494379424e-5 
3 . 87365832e-S 
2.478718S772«-4 
38366184736S-4 


10.108404109590433  I 
10.051527538954853  I 
10.329720015276905  I 
10.510348336177809  i 


Do  action  4 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8) ) ( (3  .6))) 

(((1  1) (3  1) (4  1) ) ( (2  1) (5  1) (6  1) (7  1))) 


0.009228344408 
1 . 980342e-4 
0.00415565492 
0.001719897488 


Do  action  1 


10.60308364992305  I 
10.0129417783803189  I 
10.271577166873519  I 
10.112397414823112  I 


G-4 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .8))) 

(((1  1) (6  1 ) ) ( (2  1) (3  1 ) (4  1) (5  1) (7  1))) 


2 . 22S5475021483e-4  I  0 . 026347394230459 
0.00528862635  10.62609997426044 

6 . 6109466042966e-4  I  0 . 0782644343703208 
0 . 00227466014004297  | 0 . 269288197138781 


Do  action  2 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  ,8))((3  ,C))l 

(((1  1 ) (2  1) (3  1 ) (4  1)(S  1)(6  1) (7  1))) 


1  04329425736217e-6  10.0155724159526856  I 
4 . 1985S94e-5  10.626685260821498  I 
5 . 33419053072434e-6  10.0796191804269471  I 
1 ,36332216274724e-5  10.278123142798869  I 


Do  action  2 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  ,8))((3  .6))) 

( C  Cl  1 )  (2  1)  (3  1 )  (4  1 )  (7  1)  )  (  (5  DCS  1))) 


7 . 675228716e-6  10.103992054725875  I 
3 . 8349108e-5  10.0619593967214421  I 
2 . 435437608e-4  10.329978649997262  I 
3 . 794139906e-4  10.514069898555421  I 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  CCCl  ■ 8) ) C (3  .6))) 

CCCl  9)C2  . 9) C3  . 9) C7  . 94)) C (6  l)C5  1))) 


0.156588949164407  10.114934825494697  I 
0.0687980198176093  10.050497103686616  I 
0.454476232684316  10.333581308092285  I 
0.682551962670711  10.500986762726502  | 


Do  action  4 


G-5 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  . 9) (2  .  9)  (3  . 9) (7  . 9) ) ( (5  1)(6  1))) 


0. 16075558422177 1 
0.0659511644982434 
0.450373548043819 
0.655584641801207 


10.120627158087363  I 
10.0494881816049404  I 
10.337949573828184  I 
10.491935086479512  I 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  . 9) (2  . 9) (3  . 9) (7  . 8) ) ( (5  1 ) (6  1))) 


0.171172171865179 

0.0588340261998287 

0.440116836442576 

0.588166339627446 


10.136035617389558 

10.0467571509457212 

10.349773943489843 

10.467433288174878 


Do  action  4 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  8))((3  .6))) 


(((1  . 9) (2  . 9) (3  . 9) (7  . 7) ) ( (6  l)(6  1))) 


0  181588759508587 
0.0517168879014139 
0.429860124841333 
0.520748037463686 


10.153380050152326 

10.0436829839110496 

10.363083968881515 

10.43985299705511 


Do  action  4 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8 ) ) ( ( 3  .6))) 


(((1  . 9) (2  . 9) (3  . 9) (7  . 6) ) ( (6  1)(6  1))) 


0.192005347161996 

0.0445997496029991 

0.41960341324009 

0.453329735279926 


10.17304977811234 

10.0401966762235804 

10.378178413431875 

10.408575132232206 


Do  action  4 


G-6 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8 ) ) ( ( 3  .6))) 

(((1  . 6) (2  . 6) (3  . 6) (7  . 3) ) ( (5  1)(6  1))) 


1 .34897207975795  10.222126855557463  | 
0.237394461581074  10.0390902718218029  | 
2.10897738085835  10.347272209025955  | 
2.37763665627653  10.39151066359478  I 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  0) (2  0) (3  0) (7  0) ) ( (5  1 ) (6  1))) 


0 . 0102469422046  I  0 . 608203688254405 

2 . 000350485e-4  I  0 .0118730106844199 

0 . 0045709052965  I  0 . 271304493036459 

r . 001829996545  I  0 . 108618808024717 


Do  action  1 
[19] 

Enter  evidences  observed  true  and  falsa  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8 ) ) ( (3  6))) 

(((1  . 9) (2  • 9) (3  . 9) (7  . 4) ) ( (5  1) (6  1))) 


0 .212838522438812  I  0 . 22152516286124 

0 . 0303654730061696  I  0 . 0316047878737936 

0 . 399089990037604  I  0 .415378165692666 

0 .318493130932405  I  0 . 331491883572301 


Do  action  3 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8 ) ) ( (3  .6))) 

((fl  . 8) (2  8) (3  . 8) (7  . 8) ) ( (5  1)(6  1))) 


0.531687933496788  10.127968966979455  I 
020317468238681  10.0488971455499438  I 
1.4010202101881  10.337177291636632  I 
2.01926120331513  10.48596659683397  I 


Do  action  4 


G-7 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  8))((3  .6;)) 

(((1  . 8) (2  . 8) (3  . 8) (7  . 4) ) ( (5  1)(6  1))) 


0 . 557312108814621  I  0 . 202709568277462 

0 . 104862786493507  I  0 . 0381414468S22542 

1 . 02280742971427  I  0 . 37202287412945 

1.06433095875803  i 0 . 387126110740834 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  ,8))((3  .6))/ 

(((1  . 8) (2  . 8) (3  . 8) (7  . 3) ) ( (5  1)(6  1))) 


0 . 563718152644329  I  0 . 235092619113691 

0 . 0802848125204318  I  0 . 0334319213501335 

0 . 928254234595812  I  0 . 387118488540479 

0 . 825598397618751  ! 0 . 344306970995696 


Do  action  3 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8) > ( (3  .6))) 

(((1  . 7)  (2  .  7)  (3  .7K7  .  7)  )  (  (5  1)  (6  1))) 


0.989363004696017  10.13693985305306  I 
0.348208271387984  10.0481962528307573  I 
2.4276991947236  1 0 . 33602304*650646  I 
3.4595291958953  10.478840847565547  I 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  - 8) ) C (3  .6))) 

(((1  . 7) (2  .7) (3  ,7)(7  . 4) ) ( (5  1)(6  1))) 


0.987278866756406  10.194290233750628  I 
0.204460003429965  10.0410663777867814  I 
1.7555418865292  10.352622866829263  I 
2.05125461764907  10.412020621633328  I 


Do  action  4 


G-8 


Enter  evidences  observed  true  and  falsa  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  . 1) (2  . 7) (3  . 7) (7  . 2) ) ( (6  1)(S  1))) 


0.737412281009515  10.304950786790926  | 
0.0156500274709916  10.006060977591637073  I 
1.48871234876899  10.576551843311908  I 
0.290321586199009  10.112436392305529  I 


Do  action  3 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  . 7) (2  . 7) (3  . 7) (7  . 3) ) ( (5  1)(6  1))) 


0.959917474109869  10.226943331469847 

0. 156530580777293  10.0370069^15687517 

1.5314894504644  10.362074165095921 

1 . 58182975810033  I  0 . 373975601865481 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8) ) ( (3  .6))) 

(((1  . 6) (2  . 6) (3  . 6) (7  . 3) ) ((6  1) (6  1))) 


1 . 34897207975795  I  0 . 222126855557463 

0 . 237394461581074  I  0 . 0390902718218029 

2 . 10897738085835  I  0 . 347272209025955 

2 . 37763665627653  I  0 . 39161066359478 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8)) ((3  .6))) 

(((1  . 6) (2  . 6) (3  .6) (7  . 2) ) ( (5  1)(6  1))) 


1.32256694437218  10.27053871175885  I 
0.164719834664703  10.0336943941181544  I 
1.73869706728891  10.355680533271928  I 
1.66265828487939  10.34010636085107  I 


Do  action  3 


G-9 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  . 5) (2  . 5) (3  . 5) (7  . 2) ) ( (5  1)(6  1))) 


1.62175631687925  10.268410748367406  I 
0.213908547691176  10.0354031939141498  I 
2.06639377383231  10.342001010660807  I 
2.14001056614104  10.354185047157637  I 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8) ) ( ( 3  .6))) 

(((1  . 5) (2  . 5) (3  . 5) (7  . 1) ) ( (5  l)(6  1))) 


1.57551165889084  10.352794245323212  I 
0.119531818347839  10.0267659952931259  I 
1.56163106707075  10.349686053207861  I 
1.2091342940616  10.270763706175801  I 


Do  action  1 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  . 5) (2  . 5) (3  .5) (7  . 15) ) ( (5  1) (6  1))) 


1.59863398788504  10.304273418736187  I 
0.166720183019507  10.0317324168283286  I 
1.81401242045153  10.34526712492151  I 
1.67457243010132  10.318727039514975  I 


Do  action  3 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8)) ((3  .6))) 

(((1  . 4) (2  .4) (3  .4) (7  . 2) ) ( (5  1)(6  1))) 


1.79172377066321  10.26686133740206  I 
0.246052976727611  10.0366474048714458  I 
2  22943970971684  10.33205523771785  I 
2.44684631461471  10.364436020008644  I 


Do  action  4 


G-10 


Enter  evidences  observed  true  and  lalse  and 

Belief  in  that  evidence  reading,  i.e.  (((1  ,8))((3  .6))) 


(((1  .4) (2  . 4) (3  . 4) (7  . IS)) ((5  1) (6  1))) 


1 . 76052407363S75 
0.191773540471244 
1.92992696233168 
1.91054544691939 


10.303917481021484  I 
10.0331056713278724  I 
10.33316132947617  I 
10.329815518174473  I 


Do  action  3 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 


(((1  . 3) (2  . 3) (3  . 3) (7  . 1) ) ( (5  1)(6  1))) 


1.70368008021699 

0.14027460229385 

1.54215289390464 

1.38991216565286 


10.356715460200167  I 
10.0293706077171912  I 
10.322894999851232  I 
10.29101893223141  I 


Do  action  1 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  ,8))((3  .6))) 

(((1  . 3) (2  . 3) (3  . 3) (7  . 15)) ((S  1)(6  1))) 


1.73887447380986 

0.195651701795715 

1.85445017226709 

1.93773113366571 


10.303642971011808  I 
10.0341647800985911  I 
10.323824846693397  | 
10.338387402196204  I 


Do  action  4 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8)) ((3  .6))) 

(((1  . 2) (2  . 2) (3  . 2) (7  . IB)) ((5  1)(6  1))) 


1.47762602828582 
0.17048377622691 
1.54073168638023 
1 . 68077247467094 


10.303424820415188  I 
10.0350064732544851  I 
10.316404961725444  I 
10.345163754604882  I 


Do  action  4 


G-ll 


Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  - 8) ) ( (3  .6))) 

(((1  . 2) (2  • 2) (3  . 2) (7  . 1) ) ( (6  1)(6  1))) 


1.44465888779518  10.358107647539843  I 
0.122215849529166  10.0302953387382462  I 
1.26423380050299  10.313383177207619  I 
1.20303845043113  10.298213836514292  I 


Do  action  1 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  .8))((3  .6))) 

(((1  . 1) (2  . 1) (3  . 1) (7  . 1) ) ( (5  1) (6  1))) 


0  89834091742109  10.35925449111275  I 
0.0776603828652307  10.0310570750867758  I 
0.764043631299426  10.305547816677875  I 
0.760524830642083  10.304140817122599  I 


Do  action  1 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  - 8) ) ( (3  .6))) 

(((1  0) (2  0) (3  0) (7  . 6) ) ( (5  1) (6  1))) 


0.013603012014658  10.142062846827397  I 
0.0047042602362  10.0491288694482784  I 
0.030959970828444  10.32332997724648  I 
0.0464862374883364  10.485478306478845  I 


Do  action  4 

Enter  evidences  observed  true  and  false  and 

Belief  in  that  evidence  reading,  i.e.  (((1  . 8) ) ( (3  .6))) 

(((1  . 34) (2  . 34) (3  . 34) (7  .12))((5  1)(6  1))) 


1.7526154382486  10.332865382293166  1 
0.163666368289121  10.0310843252091476  I 
1.72327651457286  10.327293188968683  I 
1.62587961434109  10.308757103529004  I 


Do  action  1 


G-12 
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